Computer Thinking NAND gates as logical basis, and perceptrons as NAND gates code · logic · thinking · compute · perceptron · neuron · sigmoid All binary logic circuits can be computed with NOT AND gates. **NAND** is the basic logic block. All other logic circuits can be built with it. A **NAND** gate evaluates binary inputs, `1` or `0`, and outputs `[ 1 | HIGH | TRUE ]` if two inputs are not the same. If the inputs are the same, it outputs `[ 0 | LOW | FALSE ]`\*. **AND** is a **NAND** gate, and the output is fed into both inputs of another **NAND** gate. An **OR** gate feeds input 1 to both inputs of one **NAND** gate, and input 2 to both inputs of another **NAND** gate. The two outputs are fed to the inputs of another **NAND** gate. A **NOR** gate is an **OR** gate, which outputs to an **AND** gate. Similarly, a correctly tuned perceptron can act as a NAND gate to form the blocks of a logic circuit of only perceptrons. ```python import math """ AND, OR, NOR, NOT, Buffer, XOR, XNOR using NAND import main >>> g11 = main.GATE(1,1) >>> g10 = main.GATE(1,0) >>> g01 = main.GATE(0,1) >>> g00 = main.GATE(0,0) >>> g11.NAND() 0 >>> g10.NAND() 1 >>> g01.NAND() 1 >>> g00.NAND() 1 """ class GATE: """All logic gates using NAND.""" def __init__(self, x1, x2): self.x1 = x1 self.x2 = x2 def NAND(self): result = int(not (self.x1 and self.x2)) return result def _NAND(self, x1, x2): result = int(not (x1 and x2)) return result def AND(self): result = self._NAND(self.x1, self.x2) return self._NAND(result, result) def OR(self): result_a = self._NAND(self.x1, self.x1) result_b = self._NAND(self.x2, self.x2) return self._NAND(result_a, result_b) def NOR(self): result_1 = self.OR() return self._NAND(result_1, result_1) def NOT(self): return (self._NAND(self.x1, self.x1), self._NAND(self.x2, self.x2)) def BUFFER(self): result_a, result_b = self.NOT() return (self._NAND(result_a, result_a), self._NAND(result_b, result_b)) def XOR(self): result1 = self._NAND(self.x1, self.x2) result2 = self._NAND(self.x1, result1) result3 = self._NAND(self.x2, result1) return self._NAND(result2, result3) def XNOR(self): result1 = self._NAND(self.x1, self.x2) result2 = self.OR() return self._NAND(result1, result2) class PERCEPTRON: """A perceptron is a simple neuron. y = wx + b. Note the similarity to the slope-intercept equation. Perceptrons are linear. Perceptrons tuned with weight -2 and bias 3 behave like NAND gates. In the NAND gate, we've added an activation. In this case, the perceptron is tuned for NAND logic. If we use a loss function to modify the weights in a training loop, the perceptron can learn. """ def __init__(self, x1, x2): self.x1 = x1 self.x2 = x2 self.weight = -2 self.bias = 3 def NAND(self): result = ((self.x1 * self.weight) + (self.x2 * self.weight)) + self.bias if result < 0: return 0 if result >= 1: return 1 def _NAND(self, x1, x2): result = ((x1 * self.weight) + (x2 * self.weight)) + self.bias if result < 0: return 0 if result >= 1: return 1 def AND(self): result = self._NAND(self.x1, self.x2) return self._NAND(result, result) # etc for all logic # Perceptron as neuron def compute(self, x): return (self.weight * x + self.bias) def activate(self, z): if z < 0: return 0 if z >= 0: return 1 # NAND gates and perceptrons form the basis of compute and the earliest vestiges # of computer thinking and learning. But perceptrons can only learn linear problems. # Non-linear problems, that is, most problems today, require non-linear neurons. class SIGMOID_NEURON: """A perceptron can become non-linear by changing the activation to a sigmoid function. The benefits go beyond linearity. Perceptrons can be extremely sensitive to weight changes, and tuning one perceptron can change the output of the entire model. The sigmoid function: σ = 1/ 1+e^-x, where x is the input, and e is Euler's number. (~2.718281828459045). It's asymptotic, so any input will output within the open interval between the 0 and 1 asymptotes. It's also differentiable, meaning we can calculate the slope at any point, which is used in backpropagation to compute each weight's contribution to the output. That slope is used by optimizers to adjust weights and induce learning. """ def __init__(self, x1, x2): self.x1 = x1 self.x2 = x2 def sigmoid(self, x): """ >>> s.sigmoid(2) 0.8807970779778823 >>> s.sigmoid(3) 0.9525741268224334 >>> s.sigmoid(4) 0.9820137900379085 >>> s.sigmoid(5) 0.9933071490757153 >>> s.sigmoid(10) 0.9999546021312976 """ return 1 / (1 + (math.exp(-1*x))) ``` The sigmoid line above, with the grouping written out, is $$\sigma(x)=\frac{1}{1+e^{-x}}$$ $e$ is about 2.718. The same curve is plotted in [Calculus](calculus.html) and used as the hidden step in [Neural nets](neural-nets.html).