One Analog Neuron Cannot Recognize Deterministic Context-Free Languages
| Autor: | Jiří Šíma, Martin Plátek |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
Class (set theory)
Chomsky hierarchy Deterministic context-free language Heaviside step function Context-free language Activation function 02 engineering and technology Combinatorics 03 medical and health sciences symbols.namesake 0302 clinical medicine Integer Computability theory 0202 electrical engineering electronic engineering information engineering symbols 020201 artificial intelligence & image processing 030217 neurology & neurosurgery Mathematics |
| Zdroj: | Neural Information Processing ISBN: 9783030367176 ICONIP (3) |
| Popis: | We analyze the computational power of discrete-time recurrent neural networks (NNs) with the saturated-linear activation function within the Chomsky hierarchy. This model restricted to integer weights coincides with binary-state NNs with the Heaviside activation function, which are equivalent to finite automata (Chomsky level 3), while rational weights make this model Turing complete even for three analog-state units (Chomsky level 0). For an intermediate model \(\alpha \)ANN of a binary-state NN that is extended with \(\alpha \ge 0\) extra analog-state neurons with rational weights, we have established the analog neuron hierarchy 0ANNs \(\subset \) 1ANNs \(\subset \) 2ANNs \(\subseteq \) 3ANNs. The separation 1ANNs \(\subsetneqq \) 2ANNs has been witnessed by the deterministic context-free language (DCFL) \(L_\#=\{0^n1^n\,|\,n\ge 1\}\) which cannot be recognized by any 1ANN even with real weights, while any DCFL (Chomsky level 2) is accepted by a 2ANN with rational weights. In this paper, we generalize this result by showing that any non-regular DCFL cannot be recognized by 1ANNs with real weights, which means (DCFLs \(\setminus \) REG\()\,\subset \,(\)2ANNs \(\setminus \) 1ANNs), implying 0ANNs \(=\) 1ANNs \(\cap \) DCFLs. For this purpose, we show that \(L_\#\) is the simplest non-regular DCFL by reducing \(L_\#\) to any language in this class, which is by itself an interesting achievement in computability theory. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|