Segmentation 
Circuits Using 
Optimization 
Constrained 
John G. Harris* 
MIT AI Lab 
545 Technology Sq., Rnl 767 
Cambridge, MA 02139 
Abstract 
A novel segmentation algorithm has been developed utilizing an absolute- 
value Slnoothness penalty instead of the more common quadratic regu- 
larizer. This functional ilnposes a piece-wise constant constraint on the 
segmented data. Since the lninilnized energy is guaranteed to be coilvex, 
there are no problems with local lninima and no complex continuation 
methods are necessary to find the unique global minimum. By interpret- 
ing the minimized energy as the generalized power of a nonlinear resistive 
network, a continuous-time analog segmentation circuit was constructed. 
1 INTRODUCTION 
Analog hardware has obvious advantages in terms of its size, speed, cost, and power 
consumption. Analog chip designers, however, should not feel constrained to map- 
ping existing digital algorithms to silicon. Many times, new algorithms must be 
adapted or invented to ensure eflqcient implelnentation in analog hardware. Novel 
analog algorithms embedded in the hardware must be simple and obey the natural 
constraints of physics. Much algorithm intuition can be gained from experimenting 
with these continuous-time nonlinear systems. For example, the algorithm described 
in this paper arose from experimentation with existing analog segmentation hard- 
ware. Surprisingly, many of these "analog" algorithms may prove useful even if a 
computer vision researcher is limited to simulating the analog hardware on a digital 
computer [7]. 
*A portion of this work is part of a Ph.D dissertation at CMtech [7]. 
797 
798 Harris 
2 ABSOLUTE-VALUE SMOOTHNESS TERM 
Rather than deal with systems that have many possible stable states, a network 
that has a unique stable state will be studied. Consider a network that minimizes: 
(.): :/ )-0 
i i 
(2) 
The absolute-vahm fimction is used for the smoothness penalty instead of tile more 
familiar quadratic term. There are two intuitive reasons why the absolute-value 
penalty is an improvement over the quadratic penalty for piece-wise constant seg- 
mentation. First, for large values of ltt i -- tti+l], the penalty is not as severe, which 
means that edges will be smoothed less. Second, small values of]u/ -- ui+l[ are 
penalized more than they are in the quadratic case, resulting in a flatter surface 
between edges. Since no complex continuation or annealing methods are necessary 
to avoid local minima, this computational model is of interest to vision researchers 
independent of any hardware implications. 
This method is very similar to constrained optiInization methods discussed by Platt 
[14] and Gill [4]. Under this interpretation, the problem is to minimize (d - 
with the constraint that tti = tti+ for all i. Equation 1 is an instance of the penalty 
method, as ,X - oc, the constraint tti = ui+i is fulfilled exactly. The absolute-value 
value penalty function given in Equat. io1 2 is an example of a nondifferential 1)enalty. 
The constraint ui = ui+l is fulfilled exactly for a finite value of ,X. However, unlike 
typical constrained optimization methods, this application requires some of these 
"exact" constraints to fail (at discontinuities) and others to be fulfilled. 
This algorithm also resembles techniques in robust statistics, a field pioneered a.nd 
formalized by Huber [9]. The need for robust estimation techniques in visual pro- 
cessing is clear since, a. single outlier may cause wild variations in standard regular- 
ization networks which rely on quadratic data constraints [17]. Rather than use the 
quadratic data constraints, robust regressiou techniques tend to limit the influence 
of outlier data point. s. 2 The absolute-value function is one method commonly used 
to reduce outlier succeptability. In fact, tile absolute-value network developed in 
this paper is a robust method if discontinuities in the data are interpreted as out- 
liers. The line process or resistive fuse networks can also be interpreted as robust 
methods using a more complex influence functions. 
3 ANALOG MODELS 
As pointed out by Poggio and Koch [15], the notion of minimizing power in linear 
networks implementing quadratic "regularized" algorithms must be replaced by the 
more general notion of minimizing the total resistor co-content [13] for nonlinear 
networks. For a voltage-controlled resistor characterized by I = f(V), the co- 
content is defined as 
I/ 
J(l/) = fo f(V')dV' (3) 
2Outlier detection techniques have been mapped to analog hardware [8]. 
Segmentation Circuits Using Constrained Optimization 799 
ll i -- 1 tl i tti + 1 
di 
Figtire 1' Nonlinear resistive network for piece-wise const, ant segmentation. 
One-dimensional stirface interpolation froin dense data will be used as the model 
problem in this paper, but these techniques generalize to sparse data in multiple 
dimensions. A standard technique for smoothing or interpolating noisy inputs di is 
to minimize an energy 1 of the form: 
_ _ 
i i 
(1) 
The first term ensures that the solution U i will be close to the data while the second 
term implements a smoothness constraint. The parameter , controls the tradeoff 
between the degree of smoothness and the fidelity to the data. Equation i can 
be interpreted as a regularization method [1] or as the power dissipated the linear 
version of the resistive network shown in Figure 1 [16]. 
Since the energy given by Equation i oversmoothes discontinuities, numerous re- 
searchers (starting with Geman and Geman [3]) have modified Equation 1 with 
line processes and successfully demonstrated piece-wise smooth segmentation. In 
these methods, the resultant energy is nonconvex and complex annealing or con- 
tinuation methods are required to converge to a good local minima of the energy 
space. This problem is solved using probabilistic [11] or deterministic annealing 
techniques [2, 10]. Line-process discontinuities have been successfully demonstrated 
in analog hardware using resistive fuse networks [5], but continuation methods are 
still required to find a good solution [6]. 
The term energy is used throughout this paper as a cost functional to be minixnized. 
It does not necessarily relate to any true energy dissipated in the real world. 
800 Harris 
(a) original image 
(c) $ = 10mV 
(b) $ = 100mV 
(d)  = lmV 
Figure 2: Various examples of tiny-tanh network simulation for varying 5. The I-V 
characteristic of the saturating resistors is I = A tanh(V/5). (a) shows a synthetic 
1.0V tower image with additive Gaussian noise of er = 0.3V which is input to the 
network. The network outputs are shown in Figures (b) 5 = 100mV, (c) 5 = 10mV 
and (d) 5 = lmV. For all simulations A = 1. 
Segmentation Circuits Using Constrained Optimization 801 
II 
Figure 3: Tiny tanh circuit. The saturating tanh characteristic is measured between 
nodes V1 and l,. Controls k aud Vc, set, the conductance and saturation voltage 
for the device. 
For a linear resistor, I = GV, the co-content is given by GV 2, which is half the 
dissipated power P = GV -. 
The absolute-value functional in Equation 2 is not strictly convex. Also, since the 
absolute-value function is nondifferentiable at the origin, hardware and software 
methods of solution will be plagued wit} instabilities and oscillations. We approx- 
imate Equation 2 with the following well-behaved convex co-content: 
1 (tt i __ di)2 +   tanh(v/6)dv (4) 
2 i i JO 
The co-content becomes the absolute-value cost function in Equation 2 in the lim- 
iting ce as 6  0. The derivative of Equation 2 yields Kirchoff's current equation 
at each node of the resistive network in Figure 1: 
(ui - di) +  tanh(Ui - ui+ ui - ui- 
6 ) +  tanh( 6 ) = 0 (5) 
Therefore, construction of this network requires a nonlinear resistor with a hyper- 
bolic tangent I-V characteristic with an extremely narrow linear region. For this 
802 Harris 
reason, t, his element is called the tiny-tanb resistor. This saturating resistor is used 
as the nonlinear element in the resistive network shown in Figure 1. Its I-V charac- 
t, eristic is I = , tanh(V/5). It, is well-known that any circuit, made of independent 
voltage sonrces and two-terlninal resistors with strictly increasing I-V characteristics 
has a. uniqne stable state. 
4 COMPUTER SIMULATIONS 
Figure 2a shows a synthetic 1.0V tower image with additive Gaussian noise of 
a = 0.3V. Fignre 2b shows the simulated result for 5 = 100mV and , = 1. As 
Mead has observed, a network of saturating resistors has a limited segmentation 
effect [12]. Unfortunately, as seen in the figtire, noise is still evident in the output, 
and the curves on either side of the step have started to slope toward one another. 
As , is increased to further smooth the noise, the two sides of the step will blend 
together into one homogeneous region. However, as the width of the linear region 
of the satnrating resistor is reduced, network segmentation properties are greatly 
enhanced. Segmentation performance improves for 5 = 10mV shown in Figure 2c 
and further improves for 5 = lmV in Figure 2d. The best segmentation occurs when 
the I-V curve resembles a step timetlon, and co-content, therefore, approximates an 
absolute-value. Decreasing 5 less than lmV shows no discernible change in the 
output. 3 
One drawback of this network is that it does not recover the exact. heights of input 
steps. Rather it, subtracts a constant fi'om the height of each input. It is straightfor- 
ward to show that the amount each uniform region is pulled towards the background 
is given by ,(perimeter/area) [7]. Significant features with large area/perimeter ra- 
rios will retain their original height. Noise points have small area/perimeter ratios 
and therefore will be pulled towards the background. Typically, the exact values of 
the heights are less important than the location of the discontinuities. Furthermore, 
it would not be difficult to construct a two-stage network to recover the exact values 
of the step heights if desired. In this scheme a tiny-tanh network would control the 
switches on a second fuse network. 
5 ANALOG IMPLEMENTATION 
Mead has constructed a CMOS saturating resistor with an I-V characteristic of the 
form I = ) tanh(V/5), where delta must be larger than 50mV because of funda- 
mental physical limitations [12]. Simulation results from section 4 suggest that for 
a tower of height b to be segmented, hi5 must be at least on the order of 1000. 
Therefore a network using Mead's saturating resistor (5 = 50mV) could segment a 
tower on the order of 50V, which is much too large a voltage to input to these chips. 
Furthermore, since we are typically interested in segmenting images into more than 
two levels even higher voltages would be required. The tiny-tanh circuit (shown in 
Figure 3) builds upon an older version of Mead's saturating resistor [18] using a 
gain stage to decrease the linear region of the device. This device can be made to 
saturate at voltages as low as 5mV. 
3These shnulations were also used to smooth and segment noisy depth data from 
correlation-based stereo algorithm run on real images [7]. 
Segmentation Circuits Using Constrained Optimization 803 
3.6' 
:2.4 
;2.0 
3.6' 
(v) 
2.4' 
2.0' 
Chip Input 
Segmented Step 
Figure 4: Measured segmentatiol performance of the tiny-tanh network for a step. 
The input, shown on the left, is about, a iV step. The output shown on the right is 
a segmented step about 0.SV in height. 
By implementing the nonlinear resistors in Figure 1 with the tiny-tanh circuit, a 
1D segmeutation network was successfully fabricated and testcd. Figure 4 shows 
the segmentation which resulted when a step (about 1V) was scanned into the chip. 
The segmented step has been reduced to about. 0.5V. No special annealing methods 
were necessary because a convex energy is being minimized. 
6 CONCLUSION 
A uovel energy functional was developed for piece-wise constant segmentation. 4 
This computational model is of interest to vision researchers independent of any 
hardware implications, because a convex energy is minimized. In sharp contrast to 
previous solutions of this problem, no complex continuation or annealing methods 
are necessary to avoid local minima. By interpreting this Lyapunov energy as the 
co-content of a nonlinear circuit, we have built and demonstrated the tiny-ta. nh 
network, a continuous-time segmentation network in analog VLSI. 
Acknowledgement s 
Mudi of this work was perform at Caltech with the support of Christof Koch and 
Carver Mead. A Hughes Aircraft graduate student fellowship and an NSF postdoc- 
toral fellowship are gratefully acknowledged. 
4This work has also been extended to segnent piece-wise linear regions, instead of the 
purely piece-wise constant processing discussed in this paper [7]. 
804 Harris 
References 
[1] 
[2] 
[3] 
[4] 
[6] 
[7] 
[8] 
[9] 
[10] 
[11] 
[12] 
[14] 
[15] 
[16] 
[17] 
[18] 
M. Bertero, T. Poggio, and V. Torre. Ill-posed problems in early vision. Proc. 
IEEE, 76:869-889, 1988. 
A. Blake and A. Zisserman. Visual Reconstruclion. MIT Press, Cambridge, 
MA, 1987. 
S. Geman and D. Geman. Stochastic relaxation, gibbs distribution and the 
bayesian restoration of images. IEEE Tras. Pallcrn An. al. Macb. Inlell., 
6:721-741, 1984. 
P. E. Gill, W. Murray, and M. H. Wright. Praclical Optimizatio,. Academic 
Press, 1981. 
J. G. Harris, C. Koch, and J. Luo. A two-dimensional analog VLSI circuit for 
detecting discontinuities in early vision. Science, 248:1209-1211, 1990. 
J. G. Harris, C. Koch, J. Luo, and J. Wyatt. Resistive fuses: analog hardware 
for detecting discontinuities in early vision. In M. Mead, C.and Ismail, editor, 
Analog VLSI Implementalton, s of Neural Syslems. Kluwer, Norwell, MA, 1989. 
J.G. Harris. Aalog models for early vision. PhD thesis, California Institute of 
Technology, Pasadena, CA, 1991. Dept. of Computation and Neural Systems. 
J.G. Harris, S.C. Liu, and B. Mathur. Discarding outliers in a nonlinear re- 
sistive network. In Iternalional Joint Conference o'n Neural Networks, pages 
501-506, Seattle, WA., July 1991. 
P.J. Huber. Robust Statistics. J. Wiley & Sons, 1981. 
C. Koch, J. Marroquin, and A. Yuille. Analog "neuronal" networks in early 
vision. Proc Natl. Acad. Sci. B. USA, 83:4263-4267, 1987. 
J. Marroquin, S. Mitter, and T. Poggio. Probabilistic solution of ill-posed 
problems in computational vision. J. Am. Statistic Assoc, 82:76-89, 1987. 
C. Mead. Analog VLSI and Neural Systems. Addison-Wesley, 1989. 
W. Millar. Some general theorems for non-linear systems possessing resistance. 
Phil. Mag., 42:1150-1160, 1951. 
J. Platt. Constraint methods for neural networks and computer graphics. Dept. 
of Computer Science Technical Report Caltech-CS-TR-89-07, California Insti- 
tute of Technology, Pasadena, CA, 1990. 
T. Poggio and C. Koch. An analog model of computation for the ill-posed prob- 
lems of early vision. Technical report, MIT Artificial Intelligence Laboratory, 
Cambridge, MA, 1984. AI Memo No. 783. 
T. Poggio and C. Koch. Ill-posed problems in early vision: fi'om computational 
theory to analogue networks. Proc. R. Soc. Lond. B, 226:303-323, 1985. 
B.C. Schunck. Robust computational vision. In Robust methods in computer 
vision workshop., 1989. 
M. A. Sivilotti, M. A. Mahowald, and C. A. Mead. Real-time visual compu- 
tation using analog CMOS processing arrays. In 1987 Stanford Conference on 
Very Large Scale Inte#ration, Cambridge, MA, 1987. MIT Press. 
