Auto Byte



Science AI




我们现在说的人工智能,很多时候指的是基于深度神经网络机器学习(或者深度学习)方法。但实际上,人工智能是一个历史悠久和丰富内涵的学科。由于这两年机器学习取得了非常好的实际效果,其它研究方向似乎被大家遗忘了。最近这种情况有点变化,似乎其它方向也在更多的发出声音。比如,前两天看到的一个新闻,“美国国防部高级计划研究局(DARPA)于不久前对Gamalon注资720万美元”。这个Gamalon就是玩“Bayesian programming”的。

正好昨天看到两篇挺有意思的文章,都是聊人工智能领域的各个“部落”(原文是tribes)。我觉得用“门派”也挺合适。虽然同在人工智能这个“武林”,他们的关系也很微妙,既有竞争,也有合作,有时还会“badmouth each other”。一篇是“AI’s Factions Get Feisty. But Really, They’re All on the Same Team”[1],第二篇是“The Many Tribes of Artificial Intelligence”[2]。特别是第二篇,还用来一张信息图形象的描述了他们之间的关系。

图片来自Intuition Machine,

这篇文章的作者非常“严肃”的给每个“部落”起了名字(当然也有的是公认的),还设计了“徽章”。我第一眼就看到了PAC Theorists那个。

下面我就搬运一下各个“部落”的说明。高亮的部分是Deep Learning,几个分支名字起的有点意思,内容也有亮点!

Symbolists - Folks who used symbolic rule-based systems to make inferences. Most of AI has revolved around this approach. The approaches that used Lisp and Prolog are in this group, as well as the SemanticWeb, RDF, and OWL. One of the most ambitious attempts at this is Doug Lenat’s Cyc that he started back in the 80’s, where he has attempted to encode in logic rules all that we understand about this world. The major flaw is the brittleness of this approach, one always seems to find edge cases where one’s rigid knowledge base doesn’t seem to apply. Reality just seems to have this kind of fuzziness and uncertainty that is inescapable. It is like playing an endless game of Whack-a-mole.


Evolutionists - Folks who apply evolutionary processes like crossover and mutation to arrive at emergent intelligent behavior. This approach is typically known as Genetic Algorithms. We do see GA techniques used in replacement of a gradient descent approach in Deep Learning, so it’s not a approach that lives in isolation. Folks in this tribe also study cellular automata such as Conway’s Game of Life [CON] and Complex Adaptive Systems (CAS).


Bayesians - Folks who use probabilistic rules and their dependencies to make inferences. Probabilistic Graph Models (PGMs) are a generalization of this approach and the primary computational mechanism is the Monte-Carlo method for sampling distributions. The approach has some similarity with the Symbolist approach in that there is a way to arrive at an explanation of the results. One other advantage of this approach is that there is a measure of uncertainty that can be expressed in the results. Edward is one library that mixes this approach with Deep Learning.

(简要翻译:Bayes流- 依靠概率去做推理,使用诸如概率图模型[Probabilistic Graph Models]和蒙特卡洛算法之类的工具。与符号主义者相类似的是,Bayes流做人工智能方法也可以在逻辑上得到解释,而且还能量化不确定性。目前有结合Bayes方法和深度学习算法的库Edward。)

Kernel Conservatives - One of the most successful methods prior to the dominance of Deep Learning was SVM. Yann LeCun calls this glorified template matching. There is what is called a kernel trick that makes an otherwise non-linear separation problem into one that is linear. Practitioners in this field live in delight over the mathematical elegance of their approach. They believe the Deep Learners are nothing but alchemists conjuring up spells without the vaguest of understanding of the consequences.

(简要翻译:Kernel保守主义者-深度学习之前,SVM是最火的算法,当时使用Kernel Trick可以把非线性的问题映射到线性平面。Kernel保守主义者对于Kernel方法的优雅性大加赞许,并且认为搞深度学习的无非就是一帮自己也不懂自己搞出来的是什么东西的炼金术士。)

Tree Huggers - Folks who use tree-based models such as Random Forests and Gradient Boosted Decision Trees. These are essentially a tree of logic rules that slice up the domain recursively to build a classifier. This approach has actually been pretty effective in many Kaggle competitions. Microsoft has an approach that melds the tree based models with Deep Learning.

(简要翻译:抱树者- 这帮人使用基于树的模型,例如随机森林,决策树等等事实上基于树的模型在Kaggle中的许多问题里很有用。微软有一个模型,融合了树模型和深度学习。)

Connectionists - Folks who believe that intelligent behavior arises from simple mechanisms that are highly interconnected. The first manifestation of this were Perceptrons back in 1959. This approach died and resurrected a few times since then. The latest incarnation is Deep Learning.

(简要翻译:联结主义者- 一群相信智能行为来源于大规模神经元互联的人。第一波是1959年的Perceptron,之后经过起起伏伏,最近一次复兴就是目前风口浪尖的深度学习联结主义内部也不是铁板一块,而是分为几个宗派:)

  • The Canadian Conspirators - Hinton, LeCun, Bengio et al. End-to-end deep learning without manual feature engineering.

    (加拿大派- Hinton,LeCun,Bengio等等,绝技是不需要手工做feature engineering的端到端学习)

  • Swiss Posse - Basically LSTM and that consciousness has been solved by two cooperating RNNs. This posse will have you lynched if you ever claim that you invented something before they did. GANs, the “coolest thing in the last 20 years” according to LeCun are also claimed to be invented by the posse.

    (瑞士帮- LSTM的提出者以及宣称使用两个互相配合的RNN就能解决意识问题的帮派。任何敢宣称自己在他们之前就发明了什么东西的人都会被瑞士帮喷到死。比如,瑞士帮最近就号称其实是他们发明了GAN)

  • British AlphaGoist - Conjecture that AI = Deep Learning + Reinforcement Learning, despite LeCun’s claim that it is just the cherry on the cake. DeepMind is one of the major proponents in this area.

    (英国狗娃- 搞出了AlphaGo的帮派,认准了AI就是深度学习加增强学习[ 虽然LeCun说增强学习不过是蛋糕上的樱桃点缀]。DeepMind是英国狗娃里面做得最出色的团队)

  • Predictive Learners - I’m using the term Yann LeCun conjured up to describe unsupervised learning. The cake of AI or the dark matter of AI. This is a major unsolved area of AI. I, however, tend to believe that the solution is in “Meta-Learning”.

    (预测主义学者- 搞无监督学习的人,根据LeCun无监督学习是AI蛋糕中最大的部分,相当于宇宙中的暗物质,也是目前尚未解决的领域)

Compressionists - Cognition and learning are compression (Actually an idea that is shared by other tribes). The origins of Information theory derives from an argument about compression. This is a universal concept that it is more powerful than the all too often abused tool of aggregate statistics.


Complexity Theorists - Employ methods coming from physics, energy-based models, complexity theory, chaos theory and statistical mechanics. Swarm AI likely fits into this category. If there’s any group that has a chance at coming up with a good explanation why Deep Learning works, then it is likely this group.

(简要翻译:复杂系统理论家- 使用从物理学,能量模型,复杂系统理论,混沌理论和统计力学等学科继承来的方法。他们最得意的作品就是Swarm AI。另外他们是最有希望能够给深度学习给出理论解释的人。)

Biological Inspirationalists - Folks who create models that are closer to what neurons appear in biology. Examples are the Numenta folks and the Spike-and-Integrate folks like IBM’s TrueNorth chip.


Connectomeist - Folks who believe that the interconnection of the brain (i.e. Connectome) is where intelligence comes from. There’s a project that is trying to replicate a virtual worm and there is some ambitious heavily funded research [HCP] that is trying to map the brain in this way.

(简要翻译:功能联结图谱论者- 认为大脑里的互相联结,即功能联结图谱,是智能的真正来源。这方面的项目包括人造蠕虫和获得大量资助的脑功能映射项目。)

Information Integration Theorists - Argue that consciou-ness emerges from some internal imagination of machines that mirrors the causality of reality. The motivation of this group is that if we are ever to understand consciousness then we have to at least start thinking about it! I, however, can’t see the relationship of learning and consciousness in their approach. It is possible that they aren’t related at all! That’s maybe why we need sleep.

(简要翻译:信息集成工程师- 认为机器意识来源于机器内部对真实世界中因果性的映射。这个团体认为我们必须首先认识“意识”的本质,才能做人工智能)

PAC Theorists - Are folks that don’t really want to discuss Artificial Intelligence, rather prefer just studying intelligence because at least they know it exists! Their whole idea is that adaptive systems perform computation expediently such that they are all probably approximately correct. In short, intelligence does not have the luxury of massive computation.

(简要翻译:PAC主义者- 这群人并不想真正讨论人工智能。他们的观点是,只要一个自适应系统能快速执行大几率近似正确的计算[probably approximately correct, PCA]就行。总而言之,智能根本不该基于大规模计算)




1. CADE METZ,“AI’s Factions Get Feisty. But Really, They’re All on the Same Team”,

2. Carlos E. Perez, “The Many Tribes of Artificial Intelligence”,



深度学习(deep learning)是机器学习的分支,是一种试图使用包含复杂结构或由多重非线性变换构成的多个处理层对数据进行高层抽象的算法。 深度学习是机器学习中一种基于对数据进行表征学习的算法,至今已有数种深度学习框架,如卷积神经网络和深度置信网络和递归神经网络等已被应用在计算机视觉、语音识别、自然语言处理、音频识别与生物信息学等领域并获取了极好的效果。




阿尔法围棋是于2014年开始由英国伦敦Google DeepMind公司开发的人工智能围棋程序。AlphaGo是第一个打败人类职业棋手的计算机程序,也是第一个打败围棋世界冠军的计算机程序,可以说是历史上最强的棋手。 技术上来说,AlphaGo的算法结合了机器学习(machine learning)和树搜索(tree search)技术,并使用了大量的人类、电脑的对弈来进行训练。AlphaGo使用蒙特卡洛树搜索(MCTS:Monte-Carlo Tree Search),以价值网络(value network)和策略网络(policy network)为指导,其中价值网络用于预测游戏的胜利者,策略网络用于选择下一步行动。价值网络和策略网络都是使用深度神经网络技术实现的,神经网络的输入是经过预处理的围棋面板的描述(description of Go board)。




在机器学习中,随机森林是一个包含多个决策树的分类器,并且其输出的类别是由个别树输出的类别的众数而定。 Leo Breiman和Adele Cutler发展出推论出随机森林的算法。而"Random Forests"是他们的商标。这个术语是1995年由贝尔实验室的Tin Kam Ho所提出的随机决策森林(random decision forests)而来的。这个方法则是结合Breimans的"Bootstrap aggregating"想法和Ho的"random subspace method" 以建造决策树的集合。


梯度下降是用于查找函数最小值的一阶迭代优化算法。 要使用梯度下降找到函数的局部最小值,可以采用与当前点的函数梯度(或近似梯度)的负值成比例的步骤。 如果采取的步骤与梯度的正值成比例,则接近该函数的局部最大值,被称为梯度上升。


映射指的是具有某种特殊结构的函数,或泛指类函数思想的范畴论中的态射。 逻辑和图论中也有一些不太常规的用法。其数学定义为:两个非空集合A与B间存在着对应关系f,而且对于A中的每一个元素x,B中总有有唯一的一个元素y与它对应,就这种对应为从A到B的映射,记作f:A→B。其中,y称为元素x在映射f下的象,记作:y=f(x)。x称为y关于映射f的原象*。*集合A中所有元素的象的集合称为映射f的值域,记作f(A)。同样的,在机器学习中,映射就是输入与输出之间的对应关系。


监督式学习(Supervised learning),是机器学习中的一个方法,可以由标记好的训练集中学到或建立一个模式(函数 / learning model),并依此模式推测新的实例。训练集是由一系列的训练范例组成,每个训练范例则由输入对象(通常是向量)和预期输出所组成。函数的输出可以是一个连续的值(称为回归分析),或是预测一个分类标签(称作分类)。


人工智能领域用逻辑来理解智能推理问题;它可以提供用于分析编程语言的技术,也可用作分析、表征知识或编程的工具。目前人们常用的逻辑分支有命题逻辑(Propositional Logic )以及一阶逻辑(FOL)等谓词逻辑。


Prolog是一种逻辑编程语言。它创建在逻辑学的理论基础之上, 最初被运用于自然语言等研究领域。现在它已广泛的应用在人工智能的研究中,它可以用来建造专家系统、自然语言理解、智能知识库等。


(人工)神经元是一个类比于生物神经元的数学计算模型,是神经网络的基本组成单元。 对于生物神经网络,每个神经元与其他神经元相连,当它“兴奋”时会向相连的神经元发送化学物质,从而改变这些神经元的电位;神经元的“兴奋”由其电位决定,当它的电位超过一个“阈值”(threshold)便会被激活,亦即“兴奋”。 目前最常见的神经元模型是基于1943年 Warren McCulloch 和 Walter Pitts提出的“M-P 神经元模型”。 在这个模型中,神经元通过带权重的连接接处理来自n个其他神经元的输入信号,其总输入值将与神经元的阈值进行比较,最后通过“激活函数”(activation function)产生神经元的输出。


在概率论和统计学中,概率图模型(probabilistic graphical model,PGM) ,简称图模型(graphical model,GM),是指一种用图结构来描述多元随机 变量之间条件独立关系的概率模型