Cogprints: No conditions. Results ordered -Date, Title.

Sentence syntax trees should be made from morphemes. Semantically ordered trees

2015-02-24T18:36:37Z

Some critique of usage of sentence parse trees in modern linguistics. Two propositions on constructing trees, as mentioned in the title. Introduction of an English-to-Tatar translator program that is being developed by the author. Precedence by specificity.

A Morse alphabet

2017-02-18T20:31:11Z

A compact alphabetic form of the Morse code is constructed following some simple rules. It is mostly useful in learning the Morse code and using it in writing form.

Targetable Named Entity Recognition in Social Media

2014-08-24T20:59:40Z

We present a novel approach for recognizing what we call targetable named entities; that is, named entities in a targeted set (e.g, movies, books, TV shows). Unlike many other NER systems that need to retrain their statistical models as new entities arrive, our approach does not require such retraining, which makes it more adaptable for types of entities that are frequently updated. For this preliminary study, we focus on one entity type, movie title, using data collected from Twitter. Our system is tested on two evaluation sets, one including only entities corresponding to movies in our training set, and the other excluding any of those entities. Our final model shows F1-scores of 76.19% and 78.70% on these evaluation sets, which gives strong evidence that our approach is completely unbiased to any particular set of entities found during training.

Neural Mechanism of Language

2017-02-18T20:25:23Z

This paper is based on our previous work on neural coding. It is a self-organized model supported by existing evidences. Firstly, we briefly introduce this model in this paper, and then we explain the neural mechanism of language and reasoning with it. Moreover, we find that the position of an area determines its importance. Specifically, language relevant areas are in the capital position of the cortical kingdom. Therefore they are closely related with autonomous consciousness and working memories. In essence, language is a miniature of the real world. Briefly, this paper would like to bridge the gap between molecule mechanism of neurons and advanced functions such as language and reasoning.

Universal Grammar Is a Universal Grammar

2014-05-10T00:08:42Z

Is Universal Grammar a universal grammar? From Chomsky's hierarchy we deduce that for each grammar there is a Turing machine, and conversely. Following this equivalence, it is immediate to conclude that a universal Turing machine is equivalent to a universal grammar. Meanwhile, in linguistics, Universal Grammar is the human brain circuitry that implements the faculty of language. So the definitive answer is achieved only when we show that the human brain is Turing complete, and that language uses this capability. So yes: Universal Grammar is a universal grammar, because the human brain circuitry that implements the faculty of language is Turing complete.

Resolution Machinery

2014-03-11T11:55:42Z

The value of syntax is controversial: some see syntax as defining us as species, while for others it just facilitates communication. To assess syntax we investigate its relation to problem resolving. First we define a problem theory from first principles, and then we translate the theory concepts to mathematics, obtaining the requirements that every resolution machine has to implement. Such a resolution machine will be able to execute any possible resolution, that is, any possible way of taking a problem expression and computing the problem solutions. Two main requirements are found: 1) syntax is needed to express problems, that is, separate words are not enough, and 2) the resolution machine has to be as powerful as lambda calculus is, that is, it has to be Turing complete. Noting that every device that can generate any possible syntax, that is, any possible syntactically correct sentence of any possible, natural or artificial, language, has to be Turing complete, we conclude that syntax and problem resolving can use the same components, as, for example, sentences, functions, and conditionals. The implication to human evolution is that syntax and problem resolving should have co-evolved in humans towards Turing completeness.

Improving the quality of Gujarati-Hindi Machine Translation through part-of-speech tagging and stemmer-assisted transliteration

2013-11-18T21:01:54Z

Machine Translation for Indian languages is an emerging research area. Transliteration is one such module that we design while designing a translation system. Transliteration means mapping of source language text into the target language. Simple mapping decreases the efficiency of overall translation system. We propose the use of stemming and part-of-speech tagging for transliteration. The effectiveness of translation can be improved if we use part-of-speech tagging and stemming assisted transliteration.We have shown that much of the content in Gujarati gets transliterated while being processed for translation to Hindi language.

Are abstract concepts like dinosaur feathers? Objectification as a conceptual tool: evidence from language and gesture of English and Polish native speakers

2013-09-17T14:26:03Z

Studies based on the Contemporary Theory of Metaphor (Lakoff & Johnson, 1980, 1999) usually identify conceptual metaphors by analysing linguistic expressions and creating a post hoc interpretation of the findings. This method has been questioned for a variety of reasons, including its circularity (Müller, 2008), lack of falsifiability (Vervaeke & Kennedy, 1996, 2004), and lack of predictive power (Ritchie, 2003). It has been argued that CTM requires additional constraints to improve its applicability for empirical research (Gibbs, 2011; Ritchie, 2003). This paper sets out to propose additional methodological structure to CTM, a theory of conceptual metaphor in which much of abstract thought is generated by metaphorical mapping from embodied experience (Ruiz de Mendoza Ibáñez & Pérez Hernández, 2011). Introducing Objectification Theory defined by Szwedek (2002, 2007, 2011) ameliorates a number of methodological issues in CTM. First, the embodiment claim of CTM in its current form cannot be empirically proven incorrect (Vervaeke & Kennedy, 2004) as any mapping within it is possible (although only some actually happen). Objectification introduces pre-metaphorical structure of the kind suggested by Glucksberg (2001), constraining source and target domain selection, predicting which mappings are more likely to happen. Second, while many claim that metaphors trace back to a literal concept based on embodied physical experience (Gibbs, Costa Lima, & Francozo, 2004), it is unclear what criteria are used to define „physical”. Metaphorical domains are often described using the terms „abstract” and „concrete”, Objectification proposes objective criteria for deciding whether a concept is experientially grounded. Finally, Objectification provides grounds for introducing a hierarchical framework for metaphor typology, preventing post-hoc addition of metaphor types if and when suitable for the explanation of a phenomenon; thus increasing the consistency of the CTM framework, both internally and with other cognitive science disciplines. This thesis focuses on providing evidence for Objectification Theory and identifying its applications in metaphor and gesture research.

The International Conference on Information and Communication Systems (ICICS 2011)

2011-09-17T17:40:15Z

he International Conference on Information and Communication Systems (ICICS 2011) is a forum for scientists, engineers, and practitioners to present their latest research results, ideas, developments, and applications in all areas of Computer and Information Sciences.

Expectations eclipsed in foreign language education: learners and educators on an ongoing journey / edited by Hülya Görür-Atabaş, Sharon Turner.

2012-11-09T18:50:42Z

Between June 2-4, 2011 Sabancı University School of Languages welcomed colleagues from 21 different countries to a collaborative exploration of the challenging and inspiring journey of learners and educators in the field of language education. The conference provided an opportunity for all stakeholders to share their views on language education. Colleagues met with world-renowned experts and authors in the fields of education and psychology, faculty and administrators from various universities and institutions, teachers from secondary educational backgrounds and higher education, as well as learners whose voices are often not directly shared but usually reported. The conference name, Eclipsing Expectations, was inspired by two natural phenomena, a solar eclipse directly before the conference, and a lunar eclipse, immediately after. Learners and educators were hereby invited to join a journey to observe, learn and exchange ideas in order

Mind: meet network. Emergence of features in conceptual metaphor.

2013-09-17T14:26:14Z

As a human product, language reflects the psychological experience of man (Radden and Dirven, 2007). One model of language and human cognition in general is connectionism, by many linguists is regarded as mathematical and, therefore, too reductive. This opinion trend seems to be reversing, however, due to the fact that many cognitive researchers begin to appreciate one attribute of network models: feature emergence. In the course of a network simulation properties emerge that were neither inbuilt nor intended by its creators (Elman, 1998), in other words, the whole becomes more than just the sum of its parts. Insight is not only drawn from the network's output, but also the means that the network utilizes to arrive at the output. It may seem obvious that the events of life should be meaningful for human beings, yet there is no widely accepted theory as to how do we derive that meaning. The most promising hypothesis regarding the question how the world is meaningful to us is that of embodied cognition (cf. Turner 2009), which postulates that the functions of the brain evolved so as to ‘understand’ the body, thus grounding the mind in an experiential foundation. Yet, the relationship between the body and the mind is far from perspicuous, as research insight is still intertwined with metaphors specific for the researcher’s methodology (Eliasmith 2003). It is the aim of this paper to investigate the conceptual metaphor in a manner that will provide some insight with regard to the role that objectification, as defined by Szwedek (2002), plays in human cognition and identify one possible consequence of embodied cognition. If the mechanism for concept formation, or categorization of the world, resembles a network, it is reasonable to assume that evidence for this is to be sought in language. Let us then postulate the existence of a network mechanism for categorization and concept formation present in the human mind and initially developed to cope with the world directly accessible to the early human (i.e. tangible). Such a network would convert external inputs to form an internal, multi modal representation of a perceived object in the brain. The sheer amount of available information and the computational restrictions of the brain would force some sort of data compression, or a computational funnel. It has been shown that a visual perception network of this kind can learn to accurately label patterns (Elman, 1998). What is more, the compression of data facilitated the recognition of prototypes of a given pattern category rather than its peripheral representations, an emergent property that supports the prototype theory of the mental lexicon (cf. Radden and Dirven, 2007). The present project proposes that, in the domain of cognition, the process of objectification, as defined by Szwedek (2002), would be an emergent property of such a system, or that if an abstract notion is computed by a neural network designed to cope with tangible concepts the data compression mechanism would require the notion to be conceptualized as an object to permit further processing. The notion of emergence of meaning from the operation of complex systems is recognised as an important process in a number of studies on metaphor comprehension. Feature emergence is said to occur when a non-salient feature of the target and the vehicle becomes highly salient in the metaphor (Utsumi 2005). Therefore, for example, should objectification emerge as a feature in the metaphor KNOWLEDGE IS A TREASURE, the metaphor would be characterised as having more features of an object than either the target or vehicle alone. This paper focuses on providing a theoretical connectionist network based on the Elman-type network (Elman, 1998) as a model of concept formation where objectification would be an emergent feature. This is followed by a psychological experiment whereby the validity of this assumption is tested through a questionnaire where two groups of participants are asked to evaluate either metaphors or their components. The model proposes an underlying relation between the mechanism for concept formation and the omnipresence of conceptual metaphors, which are interpreted as resulting from the properties of the proposed network system. Thus, an evolutionary neural mechanism is proposed for categorization of the world, that is able to cope with both concrete and abstract notions and the by-product of which are the abstract language-related phenomena, i.e. metaphors. The model presented in this paper aims at providing a unified account of how the various types of phenomena, objects, feelings etc. are categorized in the human mind, drawing on evidence from language. References: Szwedek, Aleksander. 2002. Objectification: From Object Perception To Metaphor Creation. In B. Lewandowska-Tomaszczyk and K. Turewicz (eds). Cognitive Linguistics To-day, 159-175. Frankfurt am Main: Peter Lang. Radden, Günter and Dirven, René. 2007. Cognitive English Grammar. Amsterdam/ Philadelphia: John Benjamins Publishing Company Eliasmith, Chris. 2003. Moving beyond metaphors: understanding the mind for what it is. Journal of Philosophy. C(10):493- 520. Elman, J. L. et al. 1998. Rethinking innateness: A connectionist perspective on development. Cambridge, MA: MIT Press Turner, Mark. 2009. Categorization of Time and Space Through Language. (Paper presented at the FOCUS2009 conference "Categorization of the world through language". Serock, 25-28 February 2009). Utsumi, Akira. 2005. The role of feature emergence in metaphor appreciation, Metaphor and Symbol, 20(3), 151-172.

From Simple to Complex and Ultra-complex Systems: A Paradigm Shift Towards Non-Abelian Systems Dynamics

2011-12-16T00:59:10Z

Atoms, molecules, organisms distinguish layers of reality because of the causal links that govern their behavior, both horizontally (atom-atom, molecule-molecule, organism-organism) and vertically (atom-molecule-organism). This is the first intuition of the theory of levels. Even if the further development of the theory will require imposing a number of qualifications to this initial intuition, the idea of a series of entities organized on different levels of complexity will prove correct. Living systems as well as social systems and the human mind present features remarkably different from those characterizing non-living, simple physical and chemical systems. We propose that super-complexity requires at least four different categorical frameworks, provided by the theories of levels of reality, chronotopoids, (generalized) interactions, and anticipation.

From Domains Towards a Logic of Universals: A Small Calculus for the Continuous Determination of Worlds

2010-09-13T03:57:35Z

At the end of the 19th century, 'logic' moved from the discipline of philosophy to that of mathematics. One hundred years later, we have a plethora of formal logics. Looking at the situation form informatics, the mathematical discipline proved only a temporary shelter for `logic'. For there is Domain Theory, a constructive mathematical theory which extends the notion of computability into the continuum and spans the field of all possible deductive systems. Domain Theory describes the space of data-types which computers can ideally compute -- and computation in terms of these types. Domain Theory is constructive but only potentially operational. Here one particular operational model is derived from Domain Theory which consists of `universals', that is, model independent operands and operators. With these universals, Domains (logical models) can be approximated and continuously determined. The universal data-types and rules derived from Domain Theory relate strongly to the first formal logic conceived on philosophical grounds, Aristotelian (categorical) logic. This is no accident. For Aristotle, deduction was type-dependent and he too thought in term of type independent universal `essences'. This paper initiates the next `logical' step `beyond' Domain Theory by reconnecting `formal logic' with its origin.

Power laws in Elections A Survey

2010-09-13T03:47:34Z

Empirical power laws in general elections are surveyed in Brazil, Mexico, India, and with focal analysis in Indonesia. The diversity of preference dynamics in voter’s social network and the way multi-party systems to be in its critical conditions are responsible for this pushing around the evolution of political system at general. For the special case of Indonesia, we report the existing robustness for levels of legislative elections throughout 1999 to 2009. We show that the scale free phenomena in different scales of aggregation and outline some analytical outlines and consequences that could be drawn referring to this.

Communication failure

2010-07-29T01:52:07Z

This is an encyclopedia entry and does not include an abstract.

Enhanced Trustworthy and High-Quality Information Retrieval System for Web Search Engines

2009-11-14T11:28:14Z

The WWW is the most important source of information. But, there is no guarantee for information correctness and lots of conflicting information is retrieved by the search engines and the quality of provided information also varies from low quality to high quality. We provide enhanced trustworthiness in both specific (entity) and broad (content) queries in web searching. The filtering of trustworthiness is based on 5 factors – Provenance, Authority, Age, Popularity, and Related Links. The trustworthiness is calculated based on these 5 factors and it is stored thereby increasing the performance in retrieving trustworthy websites. The calculated trustworthiness is stored only for static websites. Quality is provided based on policies selected by the user. Quality based ranking of retrieved trusted information is provided using WIQA (Web Information Quality Assessment) Framework.

A New Computational Schema for Euphonic Conjunctions in Sanskrit Processing

2009-11-14T11:28:01Z

Automated language processing is central to the drive to enable facilitated referencing of increasingly available Sanskrit E-texts. The first step towards processing Sanskrit text involves the handling of Sanskrit compound words that are an integral part of Sanskrit texts. This firstly necessitates the processing of euphonic conjunctions or sandhi-s, which are points in words or between words, at which adjacent letters coalesce and transform. The ancient Sanskrit grammarian P??ini’s codification of the Sanskrit grammar is the accepted authority in the subject. His famed s?tra-s or aphorisms, numbering approximately four thousand, tersely, precisely and comprehensively codify the rules of the grammar, including all the rules pertaining to sandhi-s. This work presents a fresh new approach to processing sandhi-s in terms of a computational schema. This new computational model is based on P??ini’s complex codification of the rules of grammar. The model has simple beginnings and is yet powerful, comprehensive and computationally lean.

Distributed Object Medical Imaging Model

2009-11-14T11:29:19Z

Abstract- Digital medical informatics and images are commonly used in hospitals today,. Because of the interrelatedness of the radiology department and other departments, especially the intensive care unit and emergency department, the transmission and sharing of medical images has become a critical issue. Our research group has developed a Java-based Distributed Object Medical Imaging Model(DOMIM) to facilitate the rapid development and deployment of medical imaging applications in a distributed environment that can be shared and used by related departments and mobile physiciansDOMIM is a unique suite of multimedia telemedicine applications developed for the use by medical related organizations. The applications support realtime patients’ data, image files, audio and video diagnosis annotation exchanges. The DOMIM enables joint collaboration between radiologists and physicians while they are at distant geographical locations. The DOMIM environment consists of heterogeneous, autonomous, and legacy resources. The Common Object Request Broker Architecture (CORBA), Java Database Connectivity (JDBC), and Java language provide the capability to combine the DOMIM resources into an integrated, interoperable, and scalable system. The underneath technology, including IDL ORB, Event Service, IIOP JDBC/ODBC, legacy system wrapping and Java implementation are explored. This paper explores a distributed collaborative CORBA/JDBC based framework that will enhance medical information management requirements and development. It encompasses a new paradigm for the delivery of health services that requires process reengineering, cultural changes, as well as organizational changes.

Lightweight Service Oriented Architecture for Pervasive Computing

2009-11-14T11:32:32Z

Pervasive computing appears like a new computing era based on networks of objects and devices evolving in a real world, radically different from distributed computing, based on networks of computers and data storages. Contrary to most context-aware approaches, we work on the assumption that pervasive software must be able to deal with a dynamic software environment before processing contextual data. After demonstrating that SOA (Service oriented Architecture) and its numerous principles are well adapted for pervasive computing, we present our extended SOA model for pervasive computing, called Service Lightweight Component Architecture (SLCA). SLCA presents various additional principles to meet completely pervasive software constraints: software infrastructure based on services for devices, local orchestrations based on lightweight component architecture and finally encapsulation of those orchestrations into composite services to address distributed composition of services. We present a sample application of the overall approach as well as some relevant measures about SLCA performances.

Implementation of Rule Based Algorithm for Sandhi-Vicheda Of Compound Hindi Words

2009-11-14T11:32:38Z

Sandhi means to join two or more words to coin new word. Sandhi literally means `putting together' or combining (of sounds), It denotes all combinatory sound-changes effected (spontaneously) for ease of pronunciation. Sandhi-vicheda describes [5] the process by which one letter (whether single or cojoined) is broken to form two words. Part of the broken letter remains as the last letter of the first word and part of the letter forms the first letter of the next letter. Sandhi-Vicheda is an easy and interesting way that can give entirely new dimension that add new way to traditional approach to Hindi Teaching. In this paper using the Rule based algorithm we have reported an accuracy of 60-80% depending upon the number of rules to be implemented.

MESURE Tool to benchmark Java Card platforms

2009-11-14T11:35:49Z

The advent of the Java Card standard has been a major turning point in smart card technology. With the growing acceptance of this standard, understanding the performance behavior of these platforms is becoming crucial. To meet this need, we present in this paper a novel benchmarking framework to test and evaluate the performance of Java Card platforms. MESURE tool is the first framework which accuracy and effectiveness are independent from the particular Java Card platform tested and CAD used.

Web Single Sign-On Authentication using SAML

2009-11-14T11:35:07Z

Companies have increasingly turned to application service providers (ASPs) or Software as a Service (SaaS) vendors to offer specialized web-based services that will cut costs and provide specific and focused applications to users. The complexity of designing, installing, configuring, deploying, and supporting the system with internal resources can be eliminated with this type of methodology, providing great benefit to organizations. However, these models can present an authentication problem for corporations with a large number of external service providers. This paper describes the implementation of Security Assertion Markup Language (SAML) and its capabilities to provide secure single sign-on (SSO) solutions for externally hosted applications.

A Constructive Mathematic approach for Natural Language formal grammars

2012-12-22T13:16:19Z

A mathematical description of natural language grammars has been proposed first by Leibniz. After the definition given by Frege of unsaturated expression and the foundation of a logical grammar by Husserl, the application of logic to treat natural language grammars in a computational way raised the interest of linguists, for example applying Lambek's categorial calculus. In recent years, the most consolidated formal grammars (e.g., Minimalism, HPSG, TAG, CCG, Dependency Grammars) began to show an interest in giving a strong psychological interpretation to the formalism and hence to natural language data on which they are applied. Nevertheless, no one seems to have paid much attention to cognitive linguistics, a branch of linguistics that actively uses concepts and results from cognitive sciences. Apparently unrelated, the study of computational concepts and formalisms has developed in pair with constructive formal systems, especially in the branch of logic called proof theory, see, e.g., the Curry-Howard isomorphism and the typed functional languages. In this paper, we want to bridge these worlds and thus present our natural language formalism, called Adpositional Grammars (AdGrams), that is founded over both cognitive linguistics and constructive mathematics.

The Usefulness of Multilevel Hash Tables with Multiple Hash Functions in Large Databases

2009-06-16T19:28:05Z

In this work, attempt is made to select three good hash functions which uniformly distribute hash values that permute their internal states and allow the input bits to generate different output bits. These functions are used in different levels of hash tables that are coded in Java Programming Language and a quite number of data records serve as primary data for testing the performances. The result shows that the two-level hash tables with three different hash functions give a superior performance over one-level hash table with two hash functions or zero-level hash table with one function in term of reducing the conflict keys and quick look-up for a particular element. The result assists to reduce the complexity of join operation in query language from O(n2) to O(1) by placing larger query result, if any, in multilevel hash tables with multiple hash functions and generate shorter query result.

Nonseparability of Shared Intentionality

2009-02-13T01:17:01Z

According to recent studies in developmental psychology and neuroscience, symbolic language is essentially intersubjective. Empathetically relating to others renders possible the acquisition of linguistic constructs. Intersubjectivity develops in early ontogenetic life when interactions between mother and infant mutually shape their relatedness. Empirical findings suggest that the shared attention and intention involved in those interactions is sustained as it becomes internalized and embodied. Symbolic language is derivative and emerges from shared intentionality. In this paper, we present a formalization of shared intentionality based upon a quantum approach. From a phenomenological viewpoint, we investigate the nonseparable, dynamic and sustainable nature of social cognition and evaluate the appropriateness of quantum interaction for modelling intersubjectivity.

The Latent Relation Mapping Engine: Algorithm and Experiments

2009-01-05T23:58:22Z

Many AI researchers and cognitive scientists have argued that analogy is the core of cognition. The most influential work on computational modeling of analogy-making is Structure Mapping Theory (SMT) and its implementation in the Structure Mapping Engine (SME). A limitation of SME is the requirement for complex hand-coded representations. We introduce the Latent Relation Mapping Engine (LRME), which combines ideas from SME and Latent Relational Analysis (LRA) in order to remove the requirement for hand-coded representations. LRME builds analogical mappings between lists of words, using a large corpus of raw text to automatically discover the semantic relations among the words. We evaluate LRME on a set of twenty analogical mapping problems, ten based on scientific analogies and ten based on common metaphors. LRME achieves human-level performance on the twenty problems. We compare LRME with a variety of alternative approaches and find that they are not able to reach the same level of performance.

A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations

2008-08-31T12:24:12Z

Recognizing analogies, synonyms, antonyms, and associations appear to be four distinct tasks, requiring distinct NLP algorithms. In the past, the four tasks have been treated independently, using a wide variety of algorithms. These four semantic classes, however, are a tiny sample of the full range of semantic phenomena, and we cannot afford to create ad hoc algorithms for each semantic phenomenon; we need to seek a unified approach. We propose to subsume a broad range of phenomena under analogies. To limit the scope of this paper, we restrict our attention to the subsumption of synonyms, antonyms, and associations. We introduce a supervised corpus-based machine learning algorithm for classifying analogous word pairs, and we show that it can solve multiple-choice SAT analogy questions, TOEFL synonym questions, ESL synonym-antonym questions, and similar-associated-both questions from cognitive psychology.

In the Beginning Was the Verb: The Emergence and Evolution of Language Problem in the Light of the Big Bang Epistemological Paradigm.

2008-01-27T03:54:50Z

The enigma of the Emergence of Natural Languages, coupled or not with the closely related problem of their Evolution is perceived today as one of the most important scientific problems. The purpose of the present study is actually to outline such a solution to our problem which is epistemologically consonant with the Big Bang solution of the problem of the Emergence of the Universe}. Such an outline, however, becomes articulable, understandable, and workable only in a drastically extended epistemic and scientific oecumene, where known and habitual approaches to the problem, both theoretical and experimental, become distant, isolated, even if to some degree still hospitable conceptual and methodological islands. The guiding light of our inquiry will be Eugene Paul Wigner's metaphor of ``the unreasonable effectiveness of mathematics in natural sciences'', i.e., the steadily evolving before our eyes, since at least XVIIth century, ``the miracle of the appropriateness of the language of mathematics for the formulation of the laws of physics''. Kurt Goedel's incompleteness and undecidability theory will be our guardian discerner against logical fallacies of otherwise apparently plausible explanations. John Bell's ``unspeakableness'' and the commonplace counterintuitive character of quantum phenomena will be our encouragers. And the radical novelty of the introduced here and adapted to our purposes Big Bang epistemological paradigm will be an appropriate, even if probably shocking response to our equally shocking discovery in the oldest among well preserved linguistic fossils of perfect mathematical structures outdoing the best artifactual Assemblers.

Emergence of Self-Organized Symbol-Based Communication in Artificial Creatures

2009-01-21T22:42:36Z

In this paper, we describe a digital scenario where we simulated the emergence of self-organized symbol-based communication among artificial creatures inhabiting a virtual world of unpredictable predatory events. In our experiment, creatures are autonomous agents that learn symbolic relations in an unsupervised manner, with no explicit feedback, and are able to engage in dynamical and autonomous communicative interactions with other creatures, even simultaneously. In order to synthesize a behavioral ecology and infer the minimum organizational constraints for the design of our creatures, we examined the well-studied case of communication in vervet monkeys. Our results show that the creatures, assuming the role of sign users and learners, behave collectively as a complex adaptive system, where self-organized communicative interactions play a major role in the emergence of symbol-based communication. We also strive in this paper for a careful use of the theoretical concepts involved, including the concepts of symbol and emergence, and we make use of a multi-level model for explaining the emergence of symbols in semiotic systems as a basis for the interpretation of inter-level relationships in the semiotic processes we are studying.

Boundary effects in a three-state modified voter model for languages

2008-01-27T03:55:57Z

The standard three-state voter model is enlarged by including the outside pressure favouring one of the three language choices and by adding some biased internal random noise. The Monte Carlo simulations are motivated by states with the population divided into three groups of various affinities to each other. We show the crucial influence of the boundaries for moderate lattice sizes like 500 x 500. By removing the fixed boundary at one side, we demonstrate that this can lead to the victory of one single choice. Noise in contrast stabilizes the choices of all three populations. In addition, we compute the persistence probability, i.e., the number of sites who have never changed their opinion during the simulation, and we consider the case of ”rigid-minded” decision makers.

Semi-Supervised Named Entity Recognition: Learning to Recognize 100 Entity Types with Little Supervision

2007-12-10T21:45:33Z

Named Entity Recognition (NER) aims to extract and to classify rigid designators in text such as proper names, biological species, and temporal expressions. There has been growing interest in this field of research since the early 1990s. In this thesis, we document a trend moving away from handcrafted rules, and towards machine learning approaches. Still, recent machine learning approaches have a problem with annotated data availability, which is a serious shortcoming in building and maintaining large-scale NER systems. In this thesis, we present an NER system built with very little supervision. Human supervision is indeed limited to listing a few examples of each named entity (NE) type. First, we introduce a proof-of-concept semi-supervised system that can recognize four NE types. Then, we expand its capacities by improving key technologies, and we apply the system to an entire hierarchy comprised of 100 NE types. Our work makes the following contributions: the creation of a proof-of-concept semi-supervised NER system; the demonstration of an innovative noise filtering technique for generating NE lists; the validation of a strategy for learning disambiguation rules using automatically identified, unambiguous NEs; and finally, the development of an acronym detection algorithm, thus solving a rare but very difficult problem in alias resolution. We believe semi-supervised learning techniques are about to break new ground in the machine learning community. In this thesis, we show that limited supervision can build complete NER systems. On standard evaluation corpora, we report performances that compare to baseline supervised systems in the task of annotating NEs in texts.

Empirical Evaluation of Four Tensor Decomposition Algorithms

2007-11-22T21:41:22Z

Higher-order tensor decompositions are analogous to the familiar Singular Value Decomposition (SVD), but they transcend the limitations of matrices (second-order tensors). SVD is a powerful tool that has achieved impressive results in information retrieval, collaborative filtering, computational linguistics, computational vision, and other fields. However, SVD is limited to two-dimensional arrays of data (two modes), and many potential applications have three or more modes, which require higher-order tensor decompositions. This paper evaluates four algorithms for higher-order tensor decomposition: Higher-Order Singular Value Decomposition (HO-SVD), Higher-Order Orthogonal Iteration (HOOI), Slice Projection (SP), and Multislice Projection (MP). We measure the time (elapsed run time), space (RAM and disk space requirements), and fit (tensor reconstruction accuracy) of the four algorithms, under a variety of conditions. We find that standard implementations of HO-SVD and HOOI do not scale up to larger tensors, due to increasing RAM requirements. We recommend HOOI for tensors that are small enough for the available RAM and MP for larger tensors.

Towards Complexity Studies of Indonesian Songs

2007-08-20Z

We see some complex properties from Indonesian music discography by means of music as perceived by Indonesian people. This covers the folk songs, national anthems, popular songs by Indonesian modern artists and performers and also from western popular and classical music as reference. The self-similarity is drawn by using the model of gyration and the internal dynamics of the pitches and durations used in songs is observed by using the logarithmic spiral model. The employed entropy model is also discussed as well as introduction to the calculated dynamic complexity of melodic structure. Some generalization on the flow of music respect to the dynamic complexity is also shown. We discover that at least there are two phases in the played song: the shorter introductory phase that ends in the peak of complexity of the song and the attenuating phase of complexity in which the multiple equilibria of the song is measured. The paper draws some interesting aspects regarding to those parameters and variables on Indonesian melodic corpora.

An Alternative Postulate to see Melody as “Language”

2007-07-14Z

The paper proposes a way to see melodic features in music/songs in the terms of “letters” constituting “words”, while in return investigating the fulfillment of Zipf-Mandelbrot Law in them. Some interesting findings are reported including some possible conjectures for classification of melodic and musical artifacts considering several aspects of culture. The paper ends with some discussions related to further directions, be it enrichment in musicology and the possible plan for musical generative art.

Conjecture to Statistical Proximity with Tree of Language (?): Report on Few Austronesian Languages of Indonesian Ethnics

2007-05-28Z

We continue some steps showing the distinctions and proximities of languages over statistical facts as it has been pioneered previously [3]. In the paper, we construct the homology tree from the distance matrix yielded from the transformation of some statistical aspects of the empirical observations into binary sequences in order to conform to the concepts of memetics [2]. The resulting visualizations show interesting facts and possibly challenge some further steps for the advancement of our understanding to the discourse of languages and ethnicities.

A Note on Ontology and Ordinary Language

2007-05-19Z

We argue for a compositional semantics grounded in a strongly typed ontology that reflects our commonsense view of the world and the way we talk about it. Assuming such a structure we show that the semantics of various natural language phenomena may become nearly trivial.

Type logic served by co-Merge, Merge and Move: an account for sluicing and questions of `common European' and Japanese types

2012-11-09T19:58:28Z

We explore the power of type-logical grammar as a linguistic theory, specifically, of a new tentative development inside the framework—a “symmetricized” Lambek Calculus, due to [Moortgat2005]. The basis for our discussion is an account we give for constructions involving questions and—in particular—involving sluicing; it seeks to solve puzzles these constructions have been setting for linguistic theory. Two things in the organization of grammar are of interest here: first, a uniform system joining structures from the surface side (syntactic) and structures from the “mind side” (discourse)—we call MERGE and co-MERGE the relations by which the former and the latter structures are arranged; second, a view on the circumstances of performing MOVE (by Syntax) from the type-logical perspective. As it is usual for type-logical grammars, the theory is conscious of semantics. We refer to examples from Japanese, on one side, and from English and Russian, on the other.

Regimes in Babel are Confirmed: Report on Findings in Several Indonesian Ethnic Biblical Texts

2007-04-04Z

The paper introduces the presence of three statistical regimes in the Zipfian analysis of texts in quantitative linguistics: the Mandelbrot, original Zipf, and Cancho- Solé-Montemurro regimes. The work is carried out over nine different languages of the same intention semantically: the bible from different languages in Indonesian ethnic and national language. As always, the same analysis is also brought in English version of the Bible for reference. The existence of the three regimes are confirmed while in advance the length of the texts are also becomes an important issue. We outline some further works regarding the quantitative analysis for parameterization used to analyze the three regimes and the task to have broad explanation, especially the microstructure of the language in human decision or linguistic effort – emerging the robustness of them.

An Observational Framework to the Zipfian Analysis among Different Languages: Studies to Indonesian Ethnic Biblical Texts

2007-04-04Z

The paper introduces the used of Zipfian statistics to observe the human languages by using the same (meaning) corpus/corpora but different in grammatical and structural utterances. We used biblical texts since they contain corpuses that have been most widely and carefully translated into many languages. The idea is to reduce the possibility of noise came from the meaning of the texts in distinctive language. The result is that the robustness of the Zipfian law is observable and some statistical differences are discovered between English and widely used national and several ethnic languages in Indonesia. The paper ends by modestly propose further possible framework in interdisciplinary approaches to human language evolution.

Creativity and the Brain

2011-05-02T15:54:22Z

Neurocognitive approach to higher cognitive functions that bridges the gap between psychological and neural level of description is introduced. Relevant facts about the brain, working memory and representation of symbols in the brain are summarized. Putative brain processes responsible for problem solving, intuition, skill learning and automatization are described. The role of non-dominant brain hemisphere in solving problems requiring insight is conjectured. Two factors seem to be essential for creativity: imagination constrained by experience, and filtering that selects most interesting solutions. Experiments with paired words association are analyzed in details and evidence for stochastic resonance effects is found. Brain activity in the process of invention of novel words is proposed as the simplest way to understand creativity using experimental and computational means. Perspectives on computational models of creativity are discussed.

Designing Domain Ontology: A Study in Lexical Semantics

2007-03-16Z

Preparing a multi-purpose lexicon requires a systematic analysis of inter-conceptual relations. These relations are of two types, namely (i) syntactic and (ii) semantic, which can further be decomposed to capture the greater explanatory adequacy. But the exploration of the lexical structure becomes intricate because of the hidden dynamics of the context; since traditionally, language has been viewed as a totality of lexicon and computation system, and major emphasis has been given to the designing of the computational system, considering the designing of the lexicon internal domain ontology as a mere metaphysical game, when in reality it is a serious epistemic concern, because of having the capacity of licensing inferences. Therefore a lexical level representation should have enough scope to incorporate the contextual information. Designing domain ontology is important since it tells us about the conceptual constellation within the coherent whole of which the related terms are meaningful. Isolating a term from the corresponding constellation will results into the evaporation of meaning. Furthermore it provides the basis, upon which the entire linguistic structure rests. If so, then how is it possible to construct a lexicon, by divorcing the ontological issues? And at the same time, ontology by itself is not enough, again because of the reason that the higher order typifications of those (grounded) concepts and their corresponding interrelations among the types ultimately results into the consequent super-ordinating levels, containing the syntactic information pertinent to a symbol manipulating system. In this paper I would show that the representation of a lexical structure, should include both kind of information which are pertinent to the closed class and as well as the open class semantics, on the basis of examples, cited from English and Bengali.

Language, logic and ontology: uncovering the structure of commonsense knowledge

2007-05-08Z

The purpose of this paper is twofold: (i) we argue that the structure of commonsense knowledge must be discovered, rather than invented; and (ii) we argue that natural language, which is the best known theory of our (shared) commonsense knowledge, should itself be used as a guide to discovering the structure of commonsense knowledge. In addition to suggesting a systematic method to the discovery of the structure of commonsense knowledge, the method we propose seems to also provide an explanation for a number of phenomena in natural language, such as metaphor, intensionality, and the semantics of nominal compounds. Admittedly, our ultimate goal is quite ambitious, and it is no less than the systematic ‘discovery’ of a well-typed ontology of commonsense knowledge, and the subsequent formulation of the longawaited goal of a meaning algebra.

Latent Semantic Indexing for Patent Information

2007-09-12Z

Latent Semantic Indexing (LSI) promises more accurate retrieval of information by incorporating statistical information on term meaning and frequency while retrieving documents as a result of a search. LSI’s precision and accuracy has been proven many times on test corpora, but the world’s patent literature poses a significant challenge in effectively implementing an LSI search engine due the size and heterogeneity of the patent corpus. Some of the factors which must be addressed to realize the goal of a more accurate patent search engine are discussed herein.

Towards Avatars with Artificial Minds: Role of Semantic Memory

2007-01-19Z

he first step towards creating avatars with human-like artificial minds is to give them human-like memory structures with an access to general knowledge about the world. This type of knowledge is stored in semantic memory. Although many approaches to modeling of semantic memories have been proposed they are not very useful in real life applications because they lack knowledge comparable to the common sense that humans have, and they cannot be implemented in a computationally efficient way. The most drastic simplification of semantic memory leading to the simplest knowledge representation that is sufficient for many applications is based on the Concept Description Vectors (CDVs) that store, for each concept, an information whether a given property is applicable to this concept or not. Unfortunately even such simple information about real objects or concepts is not available. Experiments with automatic creation of concept description vectors from various sources, including ontologies, dictionaries, encyclopedias and unstructured text sources are described. Haptek-based talking head that has an access to this memory has been created as an example of a humanized interface (HIT) that can interact with web pages and exchange information in a natural way. A few examples of applications of an avatar with semantic memory are given, including the twenty questions game and automatic creation of word puzzles.

Towards comprehensive foundations of computational intelligence

2008-01-08T00:28:59Z

Although computational intelligence (CI) covers a vast variety of different methods it still lacks an integrative theory. Several proposals for CI foundations are discussed: computing and cognition as compression, meta-learning as search in the space of data models, (dis)similarity based methods providing a framework for such meta-learning, and a more general approach based on chains of transformations. Many useful transformations that extract information from features are discussed. Heterogeneous adaptive systems are presented as particular example of transformation-based systems, and the goal of learning is redefined to facilitate creation of simpler data models. The need to understand data structures leads to techniques for logical and prototype-based rule extraction, and to generation of multiple alternative models, while the need to increase predictive power of adaptive models leads to committees of competent models. Learning from partial observations is a natural extension towards reasoning based on perceptions, and an approach to intuitive solving of such problems is presented. Throughout the paper neurocognitive inspirations are frequently used and are especially important in modeling of the higher cognitive functions. Promising directions such as liquid and laminar computing are identified and many open problems presented.

Experiments on predictability of word in context and information rate in natural language

2007-11-13T00:51:03Z

Based on data from a large-scale experiment with human subjects, we conclude that the logarithm of probability to guess a word in context (unpredictability) depends linearly on the word length. This result holds both for poetry and prose, even though with prose, the subjects don't know the length of the omitted word. We hypothesize that this effect reflects a tendency of natural language to have an even information rate.

The Missing Link between Morphemic Assemblies and Behavioral Responses:a Bayesian Information-Theoretical model of lexical processing

2006-03-16Z

The Missing Link between Morphemic Assemblies and Behavioral Responses:a Bayesian Information-Theoretical model of lexical processing

2006-03-06Z

We present the Bayesian Information-Theoretical (BIT) model of lexical processing: A mathematical model illustrating a novel approach to the modelling of language processes. The model shows how a neurophysiological theory of lexical processing relying on Hebbian association and neural assemblies can directly account for a variety of eects previously observed in behavioral experiments. We develop two information-theoretical measures of the distribution of usages of a word or morpheme. These measures are calculated through unsupervised means from corpora. We show that our measures succesfully predict responses in three visual lexical decision datasets investigating the processing of in ectional morphology in Serbian and English languages, and the eects of polysemy and homonymy in English. We discuss how our model provides a neurophysiological grounding for the facilitatory and inhibitory eects of dierent types of lexical neighborhoods. In addition, our results show how, under a model based on neural assemblies, distributed patterns of activation naturally result in the arisal of discrete symbol-like structures. Therefore, the BIT model oers a point of reconciliation in the debate between distributed connectionist and discrete localist models. Finally, we argue that the modelling framework exemplied by the BIT model, is a powerful tool for integrating the different levels of the description of the human language processing system.

Automatic Dream Sentiment Analysis

2006-08-01Z

In this position paper, we propose a first step toward automatic analysis of sentiments in dreams. 100 dreams were sampled from a dream bank created for a normative study of dreams. Two human judges assigned a score to describe dream sentiments. We ran four baseline algorithms in an attempt to automate the rating of sentiments in dreams. Particularly, we compared the General Inquirer (GI) tool, the Linguistic Inquiry and Word Count (LIWC), a weighted version of the GI lexicon and of the HM lexicon and a standard bag-of-words. We show that machine learning allows automating the human judgment with accuracy superior to majority class choice.

Cero Macareno's tablet: Exploratory data analysis around a piece of archeology (La tableta de Cero Macareno: Análisis exploratorio de datos en torno a una pieza de arqueología).

2009-01-21T22:44:26Z

Cerro Macareno’s tablet is a piece of fired clay with many incisions that pose puzzling chronological and functional queries. The information provided by the incisions has been studied with the purpose of elucidating their function. Following a coding process, several statistical techniqueshave been used. It has been proven that incisions are not randomly distributed. And while the idea that they might stand for some kind of writing loses weight, the fact that the piece may be a record of some event taking place over the time and/or space horizon is gaining importance.

Discovering Communication

2006-09-17Z

What kind of motivation drives child language development? This article presents a computational model and a robotic experiment to articulate the hypothesis that children discover communication as a result of exploring and playing with their environment. The considered robotic agent is intrinsically motivated towards situations in which it optimally progresses in learning. To experience optimal learning progress, it must avoid situations already familiar but also situations where nothing can be learnt. The robot is placed in an environment in which both communicating and non-communicating objects are present. As a consequence of its intrinsic motivation, the robot explores this environment in an organized manner focusing first on non-communicative activities and then discovering the learning potential of certain types of interactive behaviour. In this experiment, the agent ends up being interested by communication through vocal interactions without having a specific drive for communication.

Expressing Implicit Semantic Relations without Supervision

2006-08-01Z

We present an unsupervised learning algorithm that mines large text corpora for patterns that express implicit semantic relations. For a given input word pair X:Y with some unspecified semantic relations, the corresponding output list of patterns is ranked according to how well each pattern Pi expresses the relations between X and Y. For example, given X=ostrich and Y=bird, the two highest ranking output patterns are "X is the largest Y" and "Y such as the X". The output patterns are intended to be useful for finding further pairs with the same relations, to support the construction of lexicons, ontologies, and semantic networks. The patterns are sorted by pertinence, where the pertinence of a pattern Pi for a word pair X:Y is the expected relational similarity between the given pair and typical pairs for Pi. The algorithm is empirically evaluated on two tasks, solving multiple-choice SAT word analogy questions and classifying semantic relations in noun-modifier pairs. On both tasks, the algorithm achieves state-of-the-art results, performing significantly better than several alternative pattern ranking algorithms, based on tf-idf.

Similarity of Semantic Relations

2006-09-01Z

There are at least two kinds of similarity. Relational similarity is correspondence between relations, in contrast with attributional similarity, which is correspondence between attributes. When two words have a high degree of attributional similarity, we call them synonyms. When two pairs of words have a high degree of relational similarity, we say that their relations are analogous. For example, the word pair mason:stone is analogous to the pair carpenter:wood. This paper introduces Latent Relational Analysis (LRA), a method for measuring relational similarity. LRA has potential applications in many areas, including information extraction, word sense disambiguation, and information retrieval. Recently the Vector Space Model (VSM) of information retrieval has been adapted to measuring relational similarity, achieving a score of 47% on a collection of 374 college-level multiple-choice word analogy questions. In the VSM approach, the relation between a pair of words is characterized by a vector of frequencies of predefined patterns in a large corpus. LRA extends the VSM approach in three ways: (1) the patterns are derived automatically from the corpus, (2) the Singular Value Decomposition (SVD) is used to smooth the frequency data, and (3) automatically generated synonyms are used to explore variations of the word pairs. LRA achieves 56% on the 374 analogy questions, statistically equivalent to the average human score of 57%. On the related problem of classifying semantic relations, LRA achieves similar gains over the VSM.

Symbols are not uniquely human

2006-11-07Z

Modern semiotics is a branch of logics that formally defines symbol-based communication. In recent years, the semiotic classification of signs has been invoked to support the notion that symbols are uniquely human. Here we show that alarm-calls such as those used by African vervet monkeys (Cercopithecus aethiops), logically satisfy the semiotic definition of symbol. We also show that the acquisition of vocal symbols in vervet monkeys can be successfully simulated by a computer program based on minimal semiotic and neurobiological constraints. The simulations indicate that learning depends on the tutor-predator ratio, and that apprentice-generated auditory mistakes in vocal symbol interpretation have little effect on the learning rates of apprentices (up to 80% of mistakes are tolerated). In contrast, just 10% of apprentice-generated visual mistakes in predator identification will prevent any vocal symbol to be correctly associated with a predator call in a stable manner. Tutor unreliability was also deleterious to vocal symbol learning: a mere 5% of “lying” tutors were able to completely disrupt symbol learning, invariably leading to the acquisition of incorrect associations by apprentices. Our investigation corroborates the existence of vocal symbols in a non-human species, and indicates that symbolic competence emerges spontaneously from classical associative learning mechanisms when the conditioned stimuli are self-generated, arbitrary and socially efficacious. We propose that more exclusive properties of human language, such as syntax, may derive from the evolution of higher-order domains for neural association, more removed from both the sensory input and the motor output, able to support the gradual complexification of grammatical categories into syntax.

Unsupervised Named-Entity Recognition: Generating Gazetteers and Resolving Ambiguity

2006-08-01Z

In this paper, we propose a named-entity recognition (NER) system that addresses two major limitations frequently discussed in the field. First, the system requires no human intervention such as manually labeling training data or creating gazetteers. Second, the system can handle more than the three classical named-entity types (person, location, and organization). We describe the system’s architecture and compare its performance with a supervised system. We experimentally evaluate the system on a standard corpus, with the three classical named-entity types, and also on a new corpus, with a new named-entity type (car brands).

BIKE: Bilingual Keyphrase Experiments

2005-11-12Z

This paper presents a novel strategy for translating lists of keyphrases. Typical keyphrase lists appear in scientific articles, information retrieval systems and web page meta-data. Our system combines a statistical translation model trained on a bilingual corpus of scientific papers with sense-focused look-up in a large bilingual terminological resource. For the latter, we developed a novel technique that benefits from viewing the keyphrase list as contextual help for sense disambiguation. The optimal combination of modules was discovered by a genetic algorithm. Our work applies to the French / English language pair.

Computational Cognitive Models of Summarization Assessment Skills

2005-09-18Z

This paper presents a general computational cognitive model of the way a summary is assessed by teachers. It is based on models of two subprocesses: determining the importance of sentences and guessing the cognitive rules that the student may have used. All models are based on Latent Semantic Analysis, a computational model of the representation of the meaning of words and sentences. Models' performances are compared with data from an experiment conducted with 278 middle school students. The general model was implemented in a learning environment designed for helping students to write summaries.

Corpus-based Learning of Analogies and Semantic Relations

2005-08-24Z

We present an algorithm for learning from unlabeled text, based on the Vector Space Model (VSM) of information retrieval, that can solve verbal analogy questions of the kind found in the SAT college entrance exam. A verbal analogy has the form A:B::C:D, meaning "A is to B as C is to D"; for example, mason:stone::carpenter:wood. SAT analogy questions provide a word pair, A:B, and the problem is to select the most analogous word pair, C:D, from a set of five choices. The VSM algorithm correctly answers 47% of a collection of 374 college-level analogy questions (random guessing would yield 20% correct; the average college-bound senior high school student answers about 57% correctly). We motivate this research by applying it to a difficult problem in natural language processing, determining semantic relations in noun-modifier pairs. The problem is to classify a noun-modifier pair, such as "laser printer", according to the semantic relation between the noun (printer) and the modifier (laser). We use a supervised nearest-neighbour algorithm that assigns a class to a given noun-modifier pair by finding the most analogous noun-modifier pair in the training data. With 30 classes of semantic relations, on a collection of 600 labeled noun-modifier pairs, the learning algorithm attains an F value of 26.5% (random guessing: 3.3%). With 5 classes of semantic relations, the F value is 43.2% (random: 20%). The performance is state-of-the-art for both verbal analogies and noun-modifier relations.

Création de surcouche de documents hypertextes et traitement du langage naturel

2005-11-12Z

Cet article présente une extension aux algorithmes de création de surcouche de documents hypertextuels. Il s’agit de diversifier la granularité de l’information qu’il est possible de capturer en utilisant des techniques de traitement du langage naturel. Une surcouche de document Web (web page wrapper) est une vue sur des noeuds HTML contenant une information donnée et désirée. Par exemple, dans une manchette de journal, une surcouche peut baliser le nom de l’auteur, la date ou même toutes les références à un lieu ou a une compagnie quelconque. Nous avons étendu le fonctionnement d’un algorithme de création de surcouchage afin de dépasser la limite des noeuds HTML et d’extraire de l’information du contenu textuel qui s’y retrouve. Nous appliquons cette technique à la création automatique de lexiques (liste de mots).

The Emergence of Symbol-Based Communication in a Complex System of Artificial Creatures

2005-09-08Z

We present here a digital scenario to simulate the emergence of self-organized symbol-based communication among artificial creatures inhabiting a virtual world of predatory events. In order to design the environment and creatures, we seek theoretical and empirical constraints from C.S.Peirce Semiotics and an ethological case study of communication among animals. Our results show that the creatures, assuming the role of sign users and learners, behave collectively as a complex system, where self-organization of communicative interactions plays a major role in the emergence of symbol-based communication. We also strive for a careful use of the theoretical concepts involved, including the concepts of symbol, communication, and emergence, and we use a multi-level model as a basis for the interpretation of inter-level relationships in the semiotic processes we are studying.

The Emergence of Symbol-Based Communication in a Complex System of Artificial Creatures

2005-08-20Z

Emerging Linguistic Functions in Early Infancy

2006-07-23Z

This paper presents results from experimental studies on early language acquisition in infants and attempts to interpret the experimental results within the framework of the Ecological Theory of Language Acquisition (ETLA) recently proposed by (Lacerda et al., 2004a). From this perspective, the infant’s first steps in the acquisition of the ambient language are seen as a consequence of the infant’s general capacity to represent sensory input and the infant’s interaction with other actors in its immediate ecological environment. On the basis of available experimental evidence, it will be argued that ETLA offers a productive alternative to traditional descriptive views of the language acquisition process by presenting an operative model of how early linguistic function may emerge through interaction.

Measuring Semantic Similarity by Latent Relational Analysis

2005-08-11Z

This paper introduces Latent Relational Analysis (LRA), a method for measuring semantic similarity. LRA measures similarity in the semantic relations between two pairs of words. When two pairs have a high degree of relational similarity, they are analogous. For example, the pair cat:meow is analogous to the pair dog:bark. There is evidence from cognitive science that relational similarity is fundamental to many cognitive and linguistic tasks (e.g., analogical reasoning). In the Vector Space Model (VSM) approach to measuring relational similarity, the similarity between two pairs is calculated by the cosine of the angle between the vectors that represent the two pairs. The elements in the vectors are based on the frequencies of manually constructed patterns in a large corpus. LRA extends the VSM approach in three ways: (1) patterns are derived automatically from the corpus, (2) Singular Value Decomposition is used to smooth the frequency data, and (3) synonyms are used to reformulate word pairs. This paper describes the LRA algorithm and experimentally compares LRA to VSM on two tasks, answering college-level multiple-choice word analogy questions and classifying semantic relations in noun-modifier expressions. LRA achieves state-of-the-art results, reaching human-level performance on the analogy questions and significantly exceeding VSM performance on both tasks.

Scaffolding Cognition with Words

2006-07-23Z

We describe a set of experiments investigating the role of natural language symbols in scaffolding situated action. Agents are evolved to respond appropriately to commands in order to perform simple tasks. We explore three different conditions, which show a significant advantage to the re-use of a public symbol system, through self-cueing leading to qualitative changes in performance. This is modelled by looping spoken output via environment back to heard input. We argue this work can be linked to, and sheds new light on, the account of self-directed speech advanced by the developmental psychologist Vygotsky in his model of the development of higher cognitive function.

A Supervised Learning Approach to Acronym Identification

2005-06-19Z

This paper addresses the task of finding acronym-definition pairs in text. Most of the previous work on the topic is about systems that involve manually generated rules or regular expressions. In this paper, we present a supervised learning approach to the acronym identification task. Our approach reduces the search space of the supervised learning system by putting some weak constraints on the kinds of acronym-definition pairs that can be identified. We obtain results comparable to hand-crafted systems that use stronger constraints. We describe our method for reducing the search space, the features used by our supervised learning system, and our experiments with various learning schemes.

Human-Level Performance on Word Analogy Questions by Latent Relational Analysis

2004-12-11Z

This paper introduces Latent Relational Analysis (LRA), a method for measuring relational similarity. LRA has potential applications in many areas, including information extraction, word sense disambiguation, machine translation, and information retrieval. Relational similarity is correspondence between relations, in contrast with attributional similarity, which is correspondence between attributes. When two words have a high degree of attributional similarity, we call them synonyms. When two pairs of words have a high degree of relational similarity, we say that their relations are analogous. For example, the word pair mason/stone is analogous to the pair carpenter/wood; the relations between mason and stone are highly similar to the relations between carpenter and wood. Past work on semantic similarity measures has mainly been concerned with attributional similarity. For instance, Latent Semantic Analysis (LSA) can measure the degree of similarity between two words, but not between two relations. Recently the Vector Space Model (VSM) of information retrieval has been adapted to the task of measuring relational similarity, achieving a score of 47% on a collection of 374 college-level multiple-choice word analogy questions. In the VSM approach, the relation between a pair of words is characterized by a vector of frequencies of predefined patterns in a large corpus. LRA extends the VSM approach in three ways: (1) the patterns are derived automatically from the corpus (they are not predefined), (2) the Singular Value Decomposition (SVD) is used to smooth the frequency data (it is also used this way in LSA), and (3) automatically generated synonyms are used to explore reformulations of the word pairs. LRA achieves 56% on the 374 analogy questions, statistically equivalent to the average human score of 57%. On the related problem of classifying noun-modifier relations, LRA achieves similar gains over the VSM, while using a smaller corpus.

How Far Can We Go Through Social System?

2004-10-06Z

The paper elaborates an endeavor on applying the algorithmic information-theoretic computational complexity to meta-social-sciences. It is motivated by the effort on seeking the impact of the well-known incompleteness theorem to the scientific methodology approaching social phenomena. The paper uses the binary string as the model of social phenomena to gain understanding on some problems faced in the philosophy of social sciences or some traps in sociological theories. The paper ends on showing the great opportunity in recent social researches and some boundaries that limit them.

Beyond information extraction: The role of ontology in military report processing

2004-10-22Z

Information extraction tools like SMES transform natural language into formal representation, e.g. into feature structures. Doing so, these tools exploit and apply linguistic knowledge about the syntactic and morphological regularities of the language used. However, these tools apply semantic as well as pragmatic knowledge only partially at best. Automatic processing of military reports has to result in a visualization of the reports content by map as well as in an actualization of the underlying database in order to allow for the actualization of the common operational picture. Normally, however, the information provided by the result of the information extraction is not explicit enough for visualization processes and database insertions. This originates from the reports themselves that are elliptical, ambiguous, and vague. In order to overcome this obstacle, the situational context and thus semantic and pragmatic aspects have to be taken into account. In the paper at hand, we present a system that uses an ontological module to integrate semantic and pragmatic knowledge. The result of the completion contains all the specifications to allow for a visualization of the report’s content on a map as well as for a database actualization.

Combining Independent Modules in Lexical Multiple-Choice Problems

2005-01-10Z

Existing statistical approaches to natural language problems are very coarse approximations to the true complexity of language processing. As such, no single technique will be best for all problem instances. Many researchers are examining ensemble methods that combine the output of multiple modules to create more accurate solutions. This paper examines three merging rules for combining probability distributions: the familiar mixture rule, the logarithmic rule, and a novel product rule. These rules were applied with state-of-the-art results to two problems used to assess human mastery of lexical semantics -- synonym questions and analogy questions. All three merging rules result in ensembles that are more accurate than any of their component modules. The differences among the three rules are not statistically significant, but it is suggestive that the popular mixture rule is not the best rule for either of the two problems.

Connectionist Taxonomy Learning

2004-11-20Z

The paper at hand describes an approach to automatise the creation of a class taxonomy. Information about objects, e.g. "a tank is armored and moves by track", but no prior knowledge about taxonomy structure is presented to a connectionist system which organizes itself by means of activation spreading (McClelland and Rumelhart, 1981) and weight adjustments. The resulting connectionist network has a form of a taxonomy sought-after.

Creation of a Style Independent Intelligent Autonomous Citation Indexer to Support Academic Research

2004-06-05Z

This paper describes the current state of RUgle, a system for classifying and indexing papers made available on the World Wide Web, in a domain-independent and universal manner. By building RUgle with the most relaxed restrictions possible on the formatting of the documents it can process, we hope to create a system that can combine the best features of currently available closed library searches that are designed to facilitate academic research with the inclusive nature of general purpose search engines that continually crawl the web and add documents to their indexed database.

Developmental Stages of Perception and Language Acquisition in a Perceptually Grounded Robot

2005-04-14Z

The objective of this research is to develop a system for language learning based on a minimum of pre-wired language-specific functionality, that is compatible with observations of perceptual and language capabilities in the human developmental trajectory. In the proposed system, meaning (in terms of descriptions of events and spatial relations) is extracted from video images based on detection of position, motion, physical contact and their parameters. Mapping of sentence form to meaning is performed by learning grammatical constructions that are retrieved from a construction inventory based on the constellation of closed class items uniquely identifying the target sentence structure. The resulting system displays robust acquisition behavior that reproduces certain observations from developmental studies, with very modest “innate” language specificity.

Ecological Theory of Language Acquisition

2005-04-14Z

This poster outlines an Ecological Theory of Language Acquisition (ETLA). The theory views the early phases of the language acquisition process as an emergent consequence of the interaction between the infant and its linguistic environment. The newborn infant is considered to be linguistically and phonetically naïve but endowed with the ability to register a wide range of multi-sensory inputs along with the ability to detect similarity between the multi-sensory stimuli it is exposed to. The initial steps of the language acquisition process are explained as unintended and inevitable consequences of the infant’s multisensory interaction with the adult. The theoretical model deriving from ETLA is tested using the experimental data presented in the two additional contributions from our research team (Gustavsson et al, “Integration of audiovisual information in 8-months-old infants”; Lacerda, Marklund et al. “On the linguistic implications of context-bound adult-infant interactions”). The generality of the ETLA’s concept is likely to be of significance for a wide range of scientific areas, like robotics, where a central issue concerns addressing general problems of how organisms or systems might develop the ability to tap on the structure of the information embedded in their operating environments.

Embodied cognition through cultural interaction

2005-04-14Z

In this short paper we describe a robotic setup to study the self-organization of conceptualisation and language. What distinguishes this project from others is that we envision a robot with specic cognitive capacities, but without resorting to any pre-programmed representations or conceptualisations. The key to this all is self-organization and enculturation. We report preliminary results on learning motor behaviours through imitation, and sketch how the language plays a pivoting role in constructing world representations.

Frequency Value Grammar and Information Theory

2004-06-05Z

I previously laid the groundwork for Frequency Value Grammar (FVG) in papers I submitted in the proceedings of the 4th International Conference on Cognitive Science (2003), Sydney Australia, and Corpus Linguistics Conference (2003), Lancaster, UK. FVG is a formal syntax theoretically based in large part on Information Theory principles. FVG relies on dynamic physical principles external to the corpus which shape and mould the corpus whereas generative grammar and other formal syntactic theories are based exclusively on patterns (fractals) found occurring within the well-formed portion of the corpus. However, FVG should not be confused with Probability Syntax, (PS), as described by Manning (2003). PS is a corpus based approach that will yield the probability distribution of possible syntax constructions over a fixed corpus. PS makes no distinction between well and ill formed sentence constructions and assumes everything found in the corpus is well formed. In contrast, FVG’s primary objective is to distinguish between well and ill formed sentence constructions and, in so doing, relies on corpus based parameters which determine sentence competency. In PS, a syntax of high probability will not necessarily yield a well formed sentence. However, in FVG, a syntax or sentence construction of high ‘frequency value’ will yield a well-formed sentence, at least, 95% of the time satisfying most empirical standards. Moreover, in FVG, a sentence construction of ‘high frequency value’ could very well be represented by an underlying syntactic construction of low probability as determined by PS. The characteristic ‘frequency values’ calculated in FVG are not measures of probability but rather are fundamentally determined values derived from exogenous principles which impact and determine corpus based parameters serving as an index of sentence competency. The theoretical framework of FVG has broad applications beyond that of formal syntax and NLP. In this paper, I will demonstrate how FVG can be used as a model for improving the upper bound calculation of entropy of written English. Generally speaking, when a function word precedes an open class word, the backward n-gram analysis will be homomorphic with the information source and will result in frequency values more representative of co-occurrences in the information source.

From Reports to Maps

2004-11-21Z

In this paper, we will sketch a project in progress. The project aims at an application of a command and control system. The application is meant to process military reports written in natural language. It exploits computer linguistic techniques, especially information extraction and ontological augmentation. A prototype has already be completed. A real world application of report processing has to go beyond pure syntactic parsing. Semantic analysis is needed and the meaning of the report has to be constructed. Even more, the meaning has to be represented in a format such that it can be visualized within the so called ``common operational picture'' (COP). The COP is an interactive map displaying information. COP standards are provided by NATO. Since military operations of our days -- war operations as well as peace-keeping and nation building ones -- involve forces of many nations, the COP serves as main tool for synchronizing actions and plans. The paper at hand will provide some insights what kind of problems come along if language processing has to result in map visualization. It also will describe some solutions to overcome these problems.

Intelligent Content Based Title and Author Name Extraction from Formatted Documents

2004-06-05Z

This paper describes the development of algorithms for extracting the title and the names of the authors from documents available on the World Wide Web. In this paper we describe several algorithms for doing so in a manner designed not to rely on specific stylistic dictates of any document formatting standard. Rather, they are designed to rely on a combination of overt and subtle cues that form a generalized, common standard for placing this information in a document and its easy extraction by readers.

Modeling the perceptual component of conceptual learning—A coordination perspective

2004-11-29Z

Although a picture may be worth a thousand words, modeling diagrams as propositions and modeling visual processing as search through a database of verbal descriptions obscures what is problematic for the learner. Cognitive modeling of language learning and geometry has obscured the learner's problem of knowing where to look—what spaces, markings, and orientations constitute the objects of interest? Today we are launching into widespread use of multimedia instructional technology, without an adequate theory to relate perceptual processes to conceptual learning. Does this matter? In this article, I review the symbolic approach to modeling perceptual processing and show its limitations for explaining difficulties children encounter in interpreting a graphic display. I present an alternative analysis by which perceptual categorization is coupled to behavior sequences, where gesturing and emotional changes are essential for resolving impasses and breaking out of loops. I conclude by asking what kind of cognitive theory we need to exploit communication technology. Have we been correct to assume that pedagogy must be grounded in an accurate psychological model of knowledge, memory, and learning?

On the linguistic implications of context-bound adult-infant interactions

2005-04-14Z

This poster presents a study of the linguistic information potentially available in adult speech directed to 3-month-old infants. The repetitive nature of the speech directed to young infants and the ecological context of the adult-infant natural interaction setting are analyzed in the light of the “Ecological theory of language acquisition” proposed by Lacerda et al. (2004, this volume). The analysis of transcripts of adult-infant interaction sessions suggests that enough information to derive general noun associations may be available as a consequence of the particular context of the adult-infant interactions during the early stages of the language acquisition process.

Real-Time Identification of Parallel Texts from Bilingual Newsfeed

2005-06-19Z

Parallel texts are documents that present parallel translations. This paper describes a simple method that can be deployed on a real-time news feed to create an infinitely growing source of parallel texts in French and English. Our experiment was lead on the Canada Newswire news feed. Given some of its intrinsic properties, it was possible to deploy a relatively simple text matching techniques that rely on language independent cognates such numbers, capitalized words, punctuation and new lines characters. On three week of press releases, our system correctly identified the vast majority of parallel press release. It committed only minor errors on repeated news items.

Symbolic Communication in Artificial Creatures: an experiment in Artificial Life

2005-08-20Z

This is a project on Artificial Life where we simulate an ecosystem that allows cooperative interaction between agents, including intra-specific predator-warning communication in a virtual environment of predatory events. We propose, based on Peircean semiotics and informed by neuroethological constraints, an experiment to simulate the emergence of symbolic communication among artificial creatures. Here we describe the simulation environment and the creaturesrsquo control architectures, and briefly present obtained results.

Word Sense Disambiguation by Web Mining for Word Co-occurrence Probabilities

2004-07-30Z

This paper describes the National Research Council (NRC) Word Sense Disambiguation (WSD) system, as applied to the English Lexical Sample (ELS) task in Senseval-3. The NRC system approaches WSD as a classical supervised machine learning problem, using familiar tools such as the Weka machine learning software and Brill's rule-based part-of-speech tagger. Head words are represented as feature vectors with several hundred features. Approximately half of the features are syntactic and the other half are semantic. The main novelty in the system is the method for generating the semantic features, based on word co-occurrence probabilities. The probabilities are estimated using the Waterloo MultiText System with a corpus of about one terabyte of unlabeled text, collected by a web crawler.

Action Oriented Adaptive Language Games

2004-02-12Z

Anchoring of semiotic symbols

2003-07-16Z

This paper presents arguments for approaching the anchoring problem using {\em semiotic symbols}. Semiotic symbols are defined by a triadic relation between forms, meanings and referents, thus having an implicit relation to the real world.Anchors are formed between these three elements rather than between `traditional' symbols and sensory images. This allows an optimization between the form (i.e. the `traditional' symbol) and the referent. A robotic experiment based on adaptive language games illustrates how the anchoring of semiotic symbols can be achieved in a bottom-up fashion. The paper concludes that applying semiotic symbols is a potentially valuable approach toward anchoring.

Anchoring symbols to sensorimotor control

2003-07-16Z

This paper investigates how robots may emerge a lexicon to communicate complex meanings about actions such as `I am going to the red target' using simple (one-word) utterances. The main issue of the paper concerns the way these complex meanings represent the actions that are performed. It is argued that the meaning of these utterances may be represented without the need for categorising a complex flow of sensorimotor data. To illustrate the point, a simulation is presented in which robots develop such a communication system. The paper concludes by confirming that it is well possible to construct such a lexicon once robots have a number of basic sensorimotor skills available.

Aspects of Cognitive Poetics

2003-10-24Z

This paper is a short introduction to Cognitive Poetics. Cognitive poetics as I conceive of it is a far cry from what goes nowadays under the label "cognitive linguistics". Cognitive linguistics does not ask the questions this paper asks; consequently it does not answer them. In an important respect, the two approaches are even diametrically opposed. Cognitive linguistics shows very succesfully how a wide range of quite different metaphors can be reduced to the same underlying conceptual metaphor, whereas cognitive poetics makes significant distinctions between very similar metaphors, claiming that these differences make poetic expression unique. It accounts for the perceived effects of poetic texts, and relates perceived effects to poetic texts in a prin cipled manner. What is more, cognitive poetics has a lot to say about thematic, semantic, and syntactic structures, the reader's cognitive style preferring one or another "mental performance", rhyme patterns, and their interaction in generating the perce i ved effects. New Criticism, Structuralism and Formalism treated these effects, sometimes quite brilliantly, in a pre-theoretical manner. Cognitive poetics is devised to handle them in a principled manner. Finally, cognitive poetics conceives of the sema ntic and the rhythmic structure of a poem by a homogeneous set of principles. In both respects it allows for alternative (mental or vocal) performances, and handles the conflicting terms of a metaphor as well as the conflicting patterns of poetic rhythm i n conformity with the aesthetic principle of an "elegant solution to a problem": the conflicting terms of a metaphor are accommodated in a semantic interpretation; the conflicting patterns of poetic rhythm in a rhythmical performance. Above all, both semantic and rhythmic structures are shaped and constrained by cognitive processes. Cognitive Linguistics, by contrast, offers no tools for handling poetic rhythm; and objects to the Controversion Theory of metaphor. ?

Coherent Keyphrase Extraction via Web Mining

2003-08-27Z

Keyphrases are useful for a variety of purposes, including summarizing, indexing, labeling, categorizing, clustering, highlighting, browsing, and searching. The task of automatic keyphrase extraction is to select keyphrases from within the text of a given document. Automatic keyphrase extraction makes it feasible to generate keyphrases for the huge number of documents that do not have manually assigned keyphrases. A limitation of previous keyphrase extraction algorithms is that the selected keyphrases are occasionally incoherent. That is, the majority of the output keyphrases may fit together well, but there may be a minority that appear to be outliers, with no clear semantic relation to the majority or to each other. This paper presents enhancements to the Kea keyphrase extraction algorithm that are designed to increase the coherence of the extracted keyphrases. The approach is to use the degree of statistical association among candidate keyphrases as evidence that they may be semantically related. The statistical association is measured using web mining. Experiments demonstrate that the enhancements improve the quality of the extracted keyphrases. Furthermore, the enhancements are not domain-specific: the algorithm generalizes well when it is trained on one domain (computer science documents) and tested on another (physics documents).

Combining independent modules to solve multiple-choice synonym and analogy problems

2003-09-19Z

Existing statistical approaches to natural language problems are very coarse approximations to the true complexity of language processing. As such, no single technique will be best for all problem instances. Many researchers are examining ensemble methods that combine the output of successful, separately developed modules to create more accurate solutions. This paper examines three merging rules for combining probability distributions: the well known mixture rule, the logarithmic rule, and a novel product rule. These rules were applied with state-of-the-art results to two problems commonly used to assess human mastery of lexical semantics -- synonym questions and analogy questions. All three merging rules result in ensembles that are more accurate than any of their component modules. The differences among the three rules are not statistically significant, but it is suggestive that the popular mixture rule is not the best rule for either of the two problems.

A Constructive Model of Mother-Infant Interaction towards Infant’s Vowel Articulation

2004-02-12Z

Human infants seem to develop to acquire common phonemes to adults without the capability to articulate or any explicit knowledge. To understand such unrevealed human cognitive development, building a robot which reproduces such a developmental process seems effective. It will also contribute to a design principle for a robot that can communicate with human beings. This paper hypothesizes that the caregiver’s parrotry to the coo of the robot plays an important role in the phoneme acquisition process based on the implication from behavioral studies, and propose a constructive model for it. We validate the proposed model by examining whether a real robot can acquire Japanese vowels through interactions with its caregiver.

Contextual Effects on Metaphor Comprehension: Experiment and Simulation

2003-10-09Z

This paper presents a computational model of referential metaphor comprehension. This model is designed on top of Latent Semantic Analysis (LSA), a model of the representation of word and text meanings. Comprehending a referential metaphor consists in scanning the semantic neighbors of the metaphor in order to find words that are also semantically related to the context. The depth of that search is compared to the time it takes for humans to process a metaphor. In particular, we are interested in two independent variables : the nature of the reference (either a literal meaning or a figurative meaning) and the nature of the context (inductive or not inductive). We show that, for both humans and model, first, metaphors take longer to process than the literal meanings and second, an inductive context can shorten the processing time.

From Analogue to Digital Vocalizations

2004-02-13Z

Sound is a medium used by humans to carry information. The existence of this kind of medium is a pre-requisite for language. It is organized into a code, called speech, which provides a repertoire of forms that is shared in each language community. This code is necessary to support the linguistic interactions that allow humans to communicate. How then may a speech code be formed prior to the existence of linguistic interactions? Moreover, the human speech code is characterized by several properties: speech is digital and compositional (vocalizations are made of units re-used systematically in other syllables); phoneme inventories have precise regularities as well as great diversity in human languages; all the speakers of a language community categorize sounds in the same manner, but each language has its own system of categorization, possibly very different from every other. How can a speech code with these properties form? These are the questions we will approach in the paper. We will study them using the method of the artificial. We will build a society of artificial agents, and study what mechanisms may provide answers. This will not prove directly what mechanisms were used for humans, but rather give ideas about what kind of mechanism may have been used. This allows us to shape the search space of possible answers, in particular by showing what is sufficient and what is not necessary. The mechanism we present is based on a low-level model of sensory-motor interactions. We show that the integration of certain very simple and non language-specific neural devices allows a population of agents to build a speech code that has the properties mentioned above. The originality is that it pre-supposes neither a functional pressure for communication, nor the ability to have coordinated social interactions (they do not play language or imitation games). It relies on the self-organizing properties of a generic coupling between perception and production both within agents, and on the interactions between agents.

From Holistic to Discrete Speech Sounds: The Blind Snow-Flake Maker Hypothesis

2005-02-22Z

Grounded lexicon formation without explicit reference transfer: who's talking to who?

2003-07-16Z

This paper presents a first investigation regarding lexicon grounding and evolution under an iterated learning regime without an explicit transfer of reference. In the original iterated learning framework, a population contains adult speakers and learning hearers. In this paper I investigate the effects of allowing both adults and learners to take up the role of speakers and hearers with varying probabilities. The results indicate that when adults and learners can be selected as speakers and hearers, their lexicons become more similar but at the cost of reduced success in communication.

Investigating social interaction strategies for bootstrapping lexicon development

2003-07-16Z

This paper investigates how different modes of social interactions influence the bootstrapping and evolution of lexicons. This is done by comparing three language game models that differ in the type of social interactions they use. The simulations show that the language games which use either joint attention or corrective feedback as a source of contextual input are better capable of bootstrapping a lexicon than the game without such directed interactions. The simulation of the latter game, however, does show that it is possible to develop a lexicon without using directed input when the lexicon is transmitted from generation to generation.

Iterated learning and grounding: from holistic to compositional languages

2003-07-16Z

This paper presents a new computational model for studying the origins and evolution of compositional languages grounded through the interaction between agents and their environment. The model is based on previous work on adaptive grounding of lexicons and the iterated learning model. Although the model is still in a developmental phase, the first results show that a compositional language can emerge in which the structure reflects regularities present in the population's environment.

Learning Analogies and Semantic Relations

2003-07-25Z

We present an algorithm for learning from unlabeled text, based on the Vector Space Model (VSM) of information retrieval, that can solve verbal analogy questions of the kind found in the Scholastic Aptitude Test (SAT). A verbal analogy has the form A:B::C:D, meaning "A is to B as C is to D"; for example, mason:stone::carpenter:wood. SAT analogy questions provide a word pair, A:B, and the problem is to select the most analogous word pair, C:D, from a set of five choices. The VSM algorithm correctly answers 47% of a collection of 374 college-level analogy questions (random guessing would yield 20% correct). We motivate this research by relating it to work in cognitive science and linguistics, and by applying it to a difficult problem in natural language processing, determining semantic relations in noun-modifier pairs. The problem is to classify a noun-modifier pair, such as "laser printer", according to the semantic relation between the noun (printer) and the modifier (laser). We use a supervised nearest-neighbour algorithm that assigns a class to a given noun-modifier pair by finding the most analogous noun-modifier pair in the training data. With 30 classes of semantic relations, on a collection of 600 labeled noun-modifier pairs, the learning algorithm attains an F value of 26.5% (random guessing: 3.3%). With 5 classes of semantic relations, the F value is 43.2% (random: 20%). The performance is state-of-the-art for these challenging problems.

Measuring praise and criticism: Inference of semantic orientation from association

2003-09-19Z

The evaluative character of a word is called its semantic orientation. Positive semantic orientation indicates praise (e.g., "honest", "intrepid") and negative semantic orientation indicates criticism (e.g., "disturbing", "superfluous"). Semantic orientation varies in both direction (positive or negative) and degree (mild to strong). An automated system for measuring semantic orientation would have application in text classification, text filtering, tracking opinions in online discussions, analysis of survey responses, and automated chat systems (chatbots). This paper introduces a method for inferring the semantic orientation of a word from its statistical association with a set of positive and negative paradigm words. Two instances of this approach are evaluated, based on two different statistical measures of word association: pointwise mutual information (PMI) and latent semantic analysis (LSA). The method is experimentally tested with 3,596 words (including adjectives, adverbs, nouns, and verbs) that have been manually labeled positive (1,614 words) and negative (1,982 words). The method attains an accuracy of 82.8% on the full test set, but the accuracy rises above 95% when the algorithm is allowed to abstain from classifying mild words.

Robots, language, and meaning

2004-02-12Z

People use language to exchange ideas and influence the actions of others through shared conceptions of word meanings, and through a shared understanding of how word meanings are combined. Under the surface form of words lie complex networks of mental structures and processes that give rise to the richly textured semantics of natural language. Machines, in contrast, are unable to use language in human-like ways due to fundamental limitations of current computational approaches to semantic representation. To address these limitations, and to serve as a catalyst for exploring alternative approaches to language and meaning, we are developing conversational robots. The problem of endowing robots with language highlights the impossibility of isolating language from other cognitive processes. Instead, we embrace a holistic approach in which various non-linguistic elements of perception, action, and memory, provide the foundations for grounding word meaning. I will review recent results in grounding language in perception and action and sketch ongoing work for grounding a wider range of words including social terms such as "I" and "my".

THSim v3.2: The Talking Heads simulation tool

2003-07-16Z

The field of language evolution and computation may benefit from using efficient and robust simulation tools that are based on widely exploited principles within the field. The tool presented in this paper is one that could fulfil such needs. The paper presents an overview of the tool -- THSim v3.2 -- and discusses some research questions that can be investigated with it.

Vector Symbolic Architectures answer Jackendoff's challenges for cognitive neuroscience

2004-12-11Z

Jackendoff (2002) posed four challenges that linguistic combinatoriality and rules of language present to theories of brain function. The essence of these problems is the question of how to neurally instantiate the rapid construction and transformation of the compositional structures that are typically taken to be the domain of symbolic processing. He contended that typical connectionist approaches fail to meet these challenges and that the dialogue between linguistic theory and cognitive neuroscience will be relatively unproductive until the importance of these problems is widely recognised and the challenges answered by some technical innovation in connectionist modelling. This paper claims that a little-known family of connectionist models (Vector Symbolic Architectures) are able to meet Jackendoff's challenges.

Phonetic Cues and Dramatic Function Artistic Recitation of Metered Speech

2003-10-18Z

This article attempts a brief synthesis of two of my research areas: sound symbolism and poetic rhythm, focussed on Simon Russel Beale's performance of Gloucester's first soliloquy in Richard III. It explores three structural relationships between phoneti c cues and their effects: redundancy (when several phonetic cues combine to the same effect); conflicting cues (which serve to convey conflicting prosodic effects by the same stretch of speech); and overdetermination (when one phonetic cue serves to conve y a variety of unrelated -- e.g., phonological, rhythmical and expressive -- effects). Iván Fónagy speaks of dual coding of phonetic cues; the same cues convey phonological and emotive information. This article proposes "triple coding": the same cues conv ey phonological, emotive and rhythmic information. The expanded version concerns two instances of stress maxima in weak positions in Gloucester's soliloquy, performed by an outstanding British actor. One of them is the least performable kind, and this is sofar my only chance for studying it. The expansion attempts to explore a methodological innovation too: The audio version of the Merriam-Webster Dictionary offers recordings of the entries by highly trained speakers, to which the artistic reading can be compared. It may serve as a standard from which the artistic recital deviates. But this suggested to me an additional, completely unexpected possibility as well. When Cleanth Brooks speaks of irony, he means "the kind of qualification which the various elements in a context receive from the context". I suddenly realised that this allowed me to explore the kind of qualification which certain intonation contours receive from the context. .e

Quantum Computers and Quantum Computer Languages: Quantum Assembly Language and Quantum C

2002-02-09Z

We show a representation of Quantum Computers defines Quantum Turing Machines with associated Quantum Grammars. We then create examples of Quantum Grammars. Lastly we develop an algebraic approach to high level Quantum Languages using Quantum Assembly language and Quantum C language as examples.

Behavior-Based Early Language Development on a Humanoid Robot

2003-10-04Z

We are exploring the idea that early language acquisition could be better modelled on an artifcial creature by considering the pragmatic aspect of natural language and of its development in human infants. We have implemented a system of vocal behaviors on Kismet in which "words" or concepts are behaviors in a competitive hierarchy. This paper reports on the framework, the vocal system's architecture and algorithms, and some preliminary results from vocal label learning and concept formation.

Looking like a human: How conversation analytic work on gaze direction in human interaction can be relevant for design and analysis of robotic interaction.

2003-10-29Z

A crucial aspect of the development of language in children has been concerned with pragmatics - a field which explores the ways in which interaction is successfully accomplished. One aspect of this is concerned with the sequential implicativeness of our actions - that is what do particular behaviours accomplish given the specific turn by turn interactive sequence in which they occur. This paper seeks to consider some aspects of this by reference to conversation analytic work on gaze in adult interaction. In this way the paper attempts to provide a brief overview of some of the ways that our thinking about robot - human interaction can be deepened by an appreciation of conversation analytic work. In particular it argues that the empirical basis of conversation analysis (henceforth CA) offers a wonderful treasure-trove of understandings about how humans accomplish social interaction. The understanding that CA provides is derived from careful empirical scrutiny and therefore it is able to offer a perspective on interaction that is sensitive to minute detail rather than crude applications of global concepts. Thus this paper provides a provisional inspection of a small fraction of CA literature concerning the use of gaze in interaction and thinks through the relevance that this might have for the design and understanding of interacting robots. Whilst CA provides a complex understanding of human interaction, predominantly derived from the everyday talk of adults, this paper argues that the approach can provide both an idealised target of communication competence and perhaps more important a means of understanding instances of human-robot interaction. In this way CA may usefully supplement other approaches to communicative competence in work on interacting robots.

Modeling the development of lexicon with a growing self-organizing map

2002-03-25Z

We present a self-organizing neural network model that can acquire an incremental lexicon. The model allows the acquisition of new words without disrupting learned structure. The model consists of three major components. First, the word co-occurrence detector computes word transition probabilities and represents word meanings in terms of context vectors. Second, word representations are projected to a lower, constant dimension. Third, the growing lexical map (GLM) self-organizes on the dimension-reduced word representations. The model is initialized with a subset of units in GLM and a subset of the lexicon, which enables it to capture the regularities of the input space and decrease chances of catastrophic interference. During growth, new nodes are inserted in order to reduce the map quantization error, and the insertion occurs only to yet unoccupied grid positions, thus preserving the 2D map topology. We have tested GLM on a portion of parental speech extracted from the CHILDES database, with an initial 200 words scattered among 800 nodes. The model demonstrates the ability to highly preserve learned lexical structure when 100 new words are gradually added. Implications of the model are discussed with respect to language acquisition by children.

Phonemic Coding Might Result From Sensory-Motor Coupling Dynamics

2003-03-12Z

Human sound systems are invariably phonemically coded. Furthermore, phoneme inventories follow very particular tendancies. To explain these phenomena, there existed so far three kinds of approaches : ``Chomskyan''/cognitive innatism, morpho-perceptual innatism and the more recent approach of ``language as a complex cultural system which adapts under the pressure of efficient communication''. The two first approaches are clearly not satisfying, while the third, even if much more convincing, makes a lot of speculative assumptions and did not really bring answers to the question of phonemic coding. We propose here a new hypothesis based on a low-level model of sensory-motor interactions. We show that certain very simple and non language-specific neural devices allow a population of agents to build signalling systems without any functional pressure. Moreover, these systems are phonemically coded. Using a realistic vowel articulatory synthesizer, we show that the inventories of vowels have striking similarities with human vowel systems.

The physical symbol grounding problem

2003-07-16Z

This paper presents an approach to solve the symbol grounding problem within the framework of embodied cognitive science. It will be argued that symbolic structures can be used within the paradigm of embodied cognitive science by adopting an alternative definition of a symbol. In this alternative definition, the symbol may be viewed as a structural coupling between an agent's sensorimotor activations and its environment. A robotic experiment is presented in which mobile robots develop a symbolic structure from scratch by engaging in a series of language games. In this experiment it is shown that robots can develop a symbolic structure with which they can communicate the names of a few objects with a remarkable degree of success. It is further shown that, although the referents may be interpreted differently on different occasions, the objects are usually named with only one form.

Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews

2002-07-15Z

This paper presents a simple unsupervised learning algorithm for classifying reviews as recommended (thumbs up) or not recommended (thumbs down). The classification of a review is predicted by the average semantic orientation of the phrases in the review that contain adjectives or adverbs. A phrase has a positive semantic orientation when it has good associations (e.g., "subtle nuances") and a negative semantic orientation when it has bad associations (e.g., "very cavalier"). In this paper, the semantic orientation of a phrase is calculated as the mutual information between the given phrase and the word "excellent" minus the mutual information between the given phrase and the word "poor". A review is classified as recommended if the average semantic orientation of its phrases is positive. The algorithm achieves an average accuracy of 74% when evaluated on 410 reviews from Epinions, sampled from four different domains (reviews of automobiles, banks, movies, and travel destinations). The accuracy ranges from 84% for automobile reviews to 66% for movie reviews.

Towards a Theory Grounded Theory of Language

2002-05-23Z

In this paper, we build upon the idea of theory grounding and propose one specific form of theory grounding, a theory of language. Theory grounding is the idea that we can imbue our embodied artificially intelligent systems with theories by modeling the way humans, and specifically young children, develop skills with theories. Modeling theory development promises to increase the conceptual and behavioral flexibility of these systems. An example of theory development in children is the social understanding referred to as theory of mind. Language is a natural task for theory grounding because it is vital in symbolic skills and apparently necessary in developing theories. Word learning, and specifically developing a concept of words, is proposed as the first step in a theory grounded theory of language.

Towards Incremental Parsing of Natural Language using Recursive Neural Networks

2002-02-18Z

In this paper we develop novel algorithmic ideas for building a natural language parser grounded upon the hypothesis of incrementality. Although widely accepted and experimentally supported under a cognitive perspective as a model of the human parser, the incrementality assumption has never been exploited for building automatic parsers of unconstrained real texts. The essentials of the hypothesis are that words are processed in a left-to-right fashion, and the syntactic structure is kept totally connected at each step. Our proposal relies on a machine learning technique for predicting the correctness of partial syntactic structures that are built during the parsing process. A recursive neural network architecture is employed for computing predictions after a training phase on examples drawn from a corpus of parsed sentences, the Penn Treebank. Our results indicate the viability of the approach andlay out the premises for a novel generation of algorithms for natural language processing which more closely model human parsing. These algorithms may prove very useful in the development of eÆcient parsers.

Two Open Problems and a Conjecture in Mathematical Logic

2002-01-22Z

The first open problem is concerned with introducing three derivation rules in predicate calculus and the second one suggests a solution for the continuum hypothesis. The conjecture says that there will always be profound questions in any significant theory, over and above those suggested by G\"odel's incompleteness theorems.

Unsupervised Learning of Semantic Orientation from a Hundred-Billion-Word Corpus

2002-07-15Z

The evaluative character of a word is called its semantic orientation. A positive semantic orientation implies desirability (e.g., "honest", "intrepid") and a negative semantic orientation implies undesirability (e.g., "disturbing", "superfluous"). This paper introduces a simple algorithm for unsupervised learning of semantic orientation from extremely large corpora. The method involves issuing queries to a Web search engine and using pointwise mutual information to analyse the results. The algorithm is empirically evaluated using a training corpus of approximately one hundred billion words the subset of the Web that is indexed by the chosen search engine. Tested with 3,596 words (1,614 positive and 1,982 negative), the algorithm attains an accuracy of 80%. The 3,596 test words include adjectives, adverbs, nouns, and verbs. The accuracy is comparable with the results achieved by Hatzivassiloglou and McKeown (1997), using a complex four-stage supervised learning algorithm that is restricted to determining the semantic orientation of adjectives.

Simple principles for a complex output: An experiment in early syntactic development

2002-02-25Z

A set of iterative mechanisms, the Three-Step Algorithm, is proposed to account for the burst in the syntactic capacities of children over age two. These mechanisms are based on the childrens perception, memory, elementary rule-like behavior and cognitive capacities, and do not require any specific innate grammatical capacities. The relevance of the Three-Step Algorithm is tested, using the large Manchester corpus in the CHILDES database. The results show that 80% of the utterances can be exactly reconstructed and that, when incomplete reconstructions are taken into account, 94% of all utterances are reconstructed. The Three-Step Algorithm should be followed by the progressive acquisition of syntactic categories and use of slot-and-frame structures which lead to a greater and more complex linguistic mastery.

Answering Subcognitive Turing Test Questions: A Reply to French

2001-09-13Z

Robert French has argued that a disembodied computer is incapable of passing a Turing Test that includes subcognitive questions. Subcognitive questions are designed to probe the network of cultural and perceptual associations that humans naturally develop as we live, embodied and embedded in the world. In this paper, I show how it is possible for a disembodied computer to answer subcognitive questions appropriately, contrary to Frenchs claim. My approach to answering subcognitive questions is to use statistical information extracted from a very large collection of text. In particular, I show how it is possible to answer a sample of subcognitive questions taken from French, by issuing queries to a search engine that indexes about 350 million Web pages. This simple algorithm may shed light on the nature of human (sub-) cognition, but the scope of this paper is limited to demonstrating that French is mistaken: a disembodied computer can answer subcognitive questions.

Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL

2001-09-12Z

This paper presents a simple unsupervised learning algorithm for recognizing synonyms, based on statistical data acquired by querying a Web search engine. The algorithm, called PMI-IR, uses Pointwise Mutual Information (PMI) and Information Retrieval (IR) to measure the similarity of pairs of words. PMI-IR is empirically evaluated using 80 synonym test questions from the Test of English as a Foreign Language (TOEFL) and 50 synonym test questions from a collection of tests for students of English as a Second Language (ESL). On both tests, the algorithm obtains a score of 74%. PMI-IR is contrasted with Latent Semantic Analysis (LSA), which achieves a score of 64% on the same 80 TOEFL questions. The paper discusses potential applications of the new unsupervised learning algorithm and some implications of the results for LSA and LSI (Latent Semantic Indexing).

Onomatopoeia: Cuckoo-Language and Tick-Tocking+◊

2003-10-18Z

This paper is a brief phonetic investigation of the nature of onomatopoeia. Onomatopoeia is the imitation of natural noises by speech sounds. To understand this phenomenon, we must realize that there is a problem here which is by no means trivial. There i s an infinite number of noises in nature, but only twenty-something letters in an alphabet that convey in any language a closed system of about fifty (up to a maximum of 100) speech sounds. I have devoted a book length study to the expressiveness of lang u age (What Makes Sound Patterns Expressive? -- The Poetic Mode of Speech Perception), but have only fleetingly touched upon onomatopoeia. In this paper I will recapitulate from that book the issue of acoustic coding, and then will toy around with two spe ci fic cases: why does the cuckoo say "kuku" in some languages, and why the clock prefers to say "tick-tock" rather than, say, tip-top. Only fleetingly I will touch upon the question why the speech sounds [s] and [S] (S represents the initial consonant of sh oe; s the initial consonant of sue) serve generally as onomatopoeia for noise. By way of doing all this, I will discuss a higher-order issue as well: How are effects translated from reality to some semiotic system, or from one semiotic system to ano ther.U.cns

A self-organizing neural network model of the acquisition of word meaning

2001-11-23Z

In this paper we present a self-organizing connectionist model of the acquisition of word meaning. Our model consists of two neural networks and builds on the basic concepts of Hebbian learning and self-organization. One network learns to approximate word transition probabilities, which are used for lexical representation, and the other network, a self-organizing map, is trained on these representations, projecting them onto a 2D space. The model relies on lexical co-occurrence information to represent word meanings in the lexicon. The results show that our model is able to acquire semantic representations from both artificial data and real corpus of language use. In addition, the model demonstrates the ability to develop rather accurate word representations even with a sparse training set.

Anticipatory Semantic Processes

2002-06-10Z

Why anticipatory processes correspond to cognitive abilities of living systems? To be adapted to an environment, behaviors need at least i) internal representations of events occurring in the external environment; and ii) internal anticipations of possible events to occur in the external environment. Interactions of these two opposite but complementary cognitive properties lead to various patterns of experimental data on semantic processing. How to investigate dynamic semantic processes? Experimental studies in cognitive psychology offer several interests such as: i) the control of the semantic environment such as words embedded in sentences; ii) the methodological tools allowing the observation of anticipations and adapted oculomotor behavior during reading; and iii) the analyze of different anticipatory processes within the theoretical framework of semantic processing. What are the different types of semantic anticipations? Experimental data show that semantic anticipatory processes involve i) the coding in memory of sequences of words occurring in textual environments; ii) the anticipation of possible future words from currently perceived words; and iii) the selection of anticipated words as a function of the sequences of perceived words, achieved by anticipatory activations and inhibitory selection processes. How to modelize anticipatory semantic processes? Localist or distributed neural networks models can account for some types of semantic processes, anticipatory or not. Attractor neural networks coding temporal sequences are presented as good candidate for modeling anticipatory semantic processes, according to specific properties of the human brain such as i) auto-associative memory; ii) learning and memorization of sequences of patterns; and iii) anticipation of memorized patterns from previously perceived patterns.

Bootstrapping grounded symbols by minimal autonomous robots

2003-07-16Z

In this paper an experiment is presented in which two mobile robots develop a shared lexicon of which the meanings are grounded in the real world. The robots start without a lexicon nor shared meanings and play language games in which they generate new meanings and negotiate words for these meanings. The experiment tries to find the minimal conditions under which verbal communication may begin to evolve. The robots are autonomous in terms of computing and cognition, but they are otherwise far simpler than most, if not all animals. It is demonstrated that a lexicon nevertheless can be made to emerge even though there are strong limits on the size and stability of this lexicon.

Constructional Tools as the Origin of Cognitive Capacities

2000-07-04Z

It is argued that cognitive capacities can be understood as the outcome of the collective action of a set of agents created by tools that explore possible behaviours and train the agents to behave in such appropriate ways as may be discovered. The coherence of the whole system is assured by a combination of vetting the performance of new agents and dealing appropriately with any faults that the whole system may develop. This picture is shown to account for a range of cognitive capacities, including language.

Learning algorithms for keyphrase extraction

2001-09-13Z

Many academic journals ask their authors to provide a list of about five to fifteen keywords, to appear on the first page of each article. Since these key words are often phrases of two or more words, we prefer to call them keyphrases. There is a wide variety of tasks for which keyphrases are useful, as we discuss in this paper. We approach the problem of automatically extracting keyphrases from text as a supervised learning task. We treat a document as a set of phrases, which the learning algorithm must learn to classify as positive or negative examples of keyphrases. Our first set of experiments applies the C4.5 decision tree induction algorithm to this learning task. We evaluate the performance of nine different configurations of C4.5. The second set of experiments applies the GenEx algorithm to the task. We developed the GenEx algorithm specifically for automatically extracting keyphrases from text. The experimental results support the claim that a custom-designed algorithm (GenEx), incorporating specialized procedural domain knowledge, can generate better keyphrases than a general-purpose algorithm (C4.5). Subjective human evaluation of the keyphrases generated by GenEx suggests that about 80% of the keyphrases are acceptable to human readers. This level of performance should be satisfactory for a wide variety of applications.

Talking Helps: Evolving Communicating Agents for the Predator-Prey Pursuit Problem

2003-01-03Z

We analyze a general model of multi-agent communication in which all agents communicate simultaneously to a message board. A genetic algorithm is used to evolve multi-agent languages for the predator agents in a version of the predator-prey pursuit problem. We show that the resulting behavior of the communicating multi-agent system is equivalent to that of a Mealy finite state machine whose states are determined by the agents’ usage of the evolved language. Simulations show that the evolution of a communication language improves the performance of the predators. Increasing the language size (and thus increasing the number of possible states in the Mealy machine) improves the performance even further. Furthermore, the evolved communicating predators perform significantly better than all previous work on similar preys. We introduce a method for incrementally increasing the language size which results in an effective coarse-to-fine search that significantly reduces the evolution time required to find a solution. We present some observations on the effects of language size, experimental setup, and prey difficulty on the evolved Mealy machines. In particular, we observe that the start state is often revisited, and incrementally increasing the language size results in smaller Mealy machines. Finally, a simple rule is derived that provides a pessimistic estimate on the minimum language size that should be used for any multi-agent problem.

Challenging the Computational Metaphor: Implications for How We Think

1999-06-28Z

This paper explores the role of the traditional computational metaphor in our thinking as computer scientists, its influence on epistemological styles, and its implications for our understanding of cognition. It proposes to replace the conventional metaphor--a sequence of steps--with the notion of a community of interacting entities, and examines the ramifications of such a shift on these various ways in which we think.

Correlates of linguistic rhythm in the speech signal

1999-04-21Z

Spoken languages have been classified by linguists according to their rhythmic properties, and psycholinguists have relied on this classification to account for infants capacity to discriminate languages. Although researchers have measured many speech signal properties, they have failed to identify reliable acoustic characteristics for language classes. This paper presents instrumental measurements based on a consonant/vowel segmentation for eight languages. The measurements suggest that intuitive rhythm types reflect specific phonological properties, which in turn are signaled by the acoustic/phonetic properties of speech. The data support the notion of rhythm classes and also allow the simulation of infant language discrimination, consistent with the hypothesis that newborns rely on a coarse segmentation of speech. A hypothesis is proposed regarding the role of rhythm perception in language acquisition.

Learning to Extract Keyphrases from Text

2001-09-17Z

Many academic journals ask their authors to provide a list of about five to fifteen key words, to appear on the first page of each article. Since these key words are often phrases of two or more words, we prefer to call them keyphrases. There is a surprisingly wide variety of tasks for which keyphrases are useful, as we discuss in this paper. Recent commercial software, such as Microsoft?s Word 97 and Verity?s Search 97, includes algorithms that automatically extract keyphrases from documents. In this paper, we approach the problem of automatically extracting keyphrases from text as a supervised learning task. We treat a document as a set of phrases, which the learning algorithm must learn to classify as positive or negative examples of keyphrases. Our first set of experiments applies the C4.5 decision tree induction algorithm to this learning task. The second set of experiments applies the GenEx algorithm to the task. We developed the GenEx algorithm specifically for this task. The third set of experiments examines the performance of GenEx on the task of metadata generation, relative to the performance of Microsoft?s Word 97. The fourth and final set of experiments investigates the performance of GenEx on the task of highlighting, relative to Verity?s Search 97. The experimental results support the claim that a specialized learning algorithm (GenEx) can generate better keyphrases than a general-purpose learning algorithm (C4.5) and the non-learning algorithms that are used in commercial software (Word 97 and Search 97).

SARDSRN: A NEURAL NETWORK SHIFT-REDUCE PARSER

2002-12-11Z

Simple Recurrent Networks (SRNs) have been widely used in natural language tasks. SARDSRN extends the SRN by explicitly representing the input sequence in a SARDNET self-organizing map. The distributed SRN component leads to good generalization and robust cognitive properties, whereas the SARDNET map provides exact representations of the sentence constituents. This combination allows SARDSRN to learn to parse sentences with more complicated structure than can the SRN alone, and suggests that the approach could scale up to realistic natural language.

Working with Constrained Systems: A Review of A. K. Joshi's IJCAI-97 Research Excellence Award Acceptance Lecture

2000-11-12Z

This is a brief review of Joshi's award acceptance lecture published in AI Magazine. This review appeared in the AI Watch column in Computers and Society, a quarterly magazine.

What sort of architecture is required for a human-like agent?

1998-06-22Z

This paper is about how to give human-like powers to complete agents. For this the most important design choice concerns the overall architecture. Questions regarding detailed mechanisms, forms of representations, inference capabilities, knowledge etc. are best addressed in the context of a global architecture in which different design decisions need to be linked. Such a design would assemble various kinds of functionality into a complete coherent working system, in which there are many concurrent, partly independent, partly mutually supportive, partly potentially incompatible processes, addressing a multitude of issues on different time scales, including asynchronous, concurrent, motive generators. Designing human like agents is part of the more general problem of understanding design space, niche space and their interrelations, for, in the abstract, there is no one optimal design, as biological diversity on earth shows.

Disambiguation and Grammar as Emergent Soft Constraints

1999-01-03Z

When reading a sentence such as "The diplomat threw the ball in the ballpark for the princess" our interpretation changes from a dance event to baseball and back to dance. Such on-line disambiguation happens automatically and appears to be based on dynamically combining the strengths of association between the keywords and the two senses. Subsymbolic neural networks are very good at modeling such behavior. They learn word meanings as soft constraints on interpretation, and dynamically combine these constraints to form the most likely interpretation. On the other hand, it is very difficult to show how systematic language structures such as relative clauses could be processed in such a system. The network would only learn to associate them to specific contexts and would not be able to process new combinations of them. A closer look at understanding embedded clauses shows that humans are not very systematic in processing grammatical structures either. For example, "The girl who the boy who the girl who lived next door blamed hit cried" is very difficult to understand, whereas "The car that the man who the dog that had rabies bit drives is in the garage" is not. This difference emerges from the same semantic constraints that are at work in the disambiguation task. In this chapter we will show how the subsymbolic parser can be combined with high-level control that allows the system to process novel combinations of relative clauses systematically, while still being sensitive to the semantic constraints.

The Representation of Space in Mental Models derived from Text

1998-04-01Z

Mental models of text are representations of what the text is about (i.e., situations), rather than representations of the text itself. Many mental model theories propose that mental models, like real situations, are played out in a medium analogous to a Euclidean space so that distance has functional consequences. For example, when mentally manipulating one element of the representation, other elements that are spatially close will be noticed and their short-term accessibility enhanced. To test this noticing hypothesis, participants read texts that described the object by object construction of a spatial layout. According to the text, a critical object ended up close to a target object (in the spatial layout) or far from the target object. In neither case, however, was the relation between the critical object and the target object explicitly described in the text. The noticing hypothesis predicts that the accessibility of the target object will be enhanced when the critical object is close to it. We tested this prediction in seven experiments in which we also manipulated the number of objects described, whether the description was accompanied by a diagram, the presentation modality of the description, the number of dimensions in the spatial layout, and the measurement of accessibility. We failed to find consistent support for the noticing hypothesis. The data compel the conclusions that a) spatial representations can be formed when reading, b) these representations do not support automatic noticing of implicit spatial relations, c) it is likely that the spatial representation is more topological or functional than Euclidean.

Text and Discourse Understanding: The DISCERN System

1999-01-03Z

The subsymbolic approach to natural language processing (NLP) captures a number of intriguing properties of human-like information processing such as learning from examples, context sensitivity, generalization, robustness of behavior, and intuitive reasoning. Within this new paradigm, the central issues are quite different from (even incompatible with) the traditional issues in symbolic NLP, and the research has proceeded without much in common with the past. However, the ultimate goal is still the same: to understand how humans process language. Even if NLP is being built on a new foundation, as can be argued, many of the results obtained through symbolic research are still valid, and could be used as a guide for developing subsymbolic models of natural language processing. This is where DISCERN (DIstributed SCript processing and Episodic memoRy Network (Miikkulainen 1993), a subsymbolic neural network model of script-based story understanding, fits in. DISCERN is purely a subsymbolic model, but at the high level it consists of modules and information structures similar to those of symbolic systems, such as scripts, lexicon, and episodic memory. At the highest level of natural language processing such as text and discourse understanding, the symbolic and subsymbolic paradigms have to address the same basic issues. Outlining a subsymbolic approach to those issues is the purpose of DISCERN. In more specific terms, DISCERN aims: (1) to demonstrate that distributed artificial neural networks can be used to build a large-scale natural language processing system that performs approximately at the level of symbolic models; (2) to show that several cognitive phenomena can be explained at the subsymbolic level using the special properties of these networks; and (3) to identify central issues in subsymbolic NLP and to develop well-motivated techniques to deal with them. To the extent that DISCERN is successful in these areas, it constitutes a first step towards building text and discourse understanding systems within the subsymbolic paradigm.

Thought as word dynamics

1999-06-27Z

A Hebbian model for speech generation opens a number of paths. A cross-linguistic scheme of functional relationships (inspired by Aristotle) dispenses with distraction by the "parts of speech" distinctions, while bridging the gap between "contents" and "structure" words. A gradient model identifies emotional and rational dynamics and shows speech generation as a process where a speaker's dissatisfaction gets minimised.

The Use of Situation Theory in Context Modeling

1998-06-16Z

At the heart of natural language processing is the understanding of context dependent meanings. This paper presents a preliminary model of formal contexts based on situation theory. It also gives a worked-out example to show the use of contexts in lifting, i.e., how propositions holding in a particular context transform when they are moved to another context. This is useful in NLP applications where preserving meaning is a desideratum.

Program Semantics and Classical Logic

2006-08-06Z

In the tradition of Denotational Semantics one usually lets program constructs take their denotations in reflexive domains, i.e. in domains where self-application is possible. For the bulk of programming constructs, however, working with reflexive domains is an unnecessary complication. In this paper we shall use the domains of ordinary classical type logic to provide the semantics of a simple programming language containing choice and recursion. We prove that the rule of {\em Scott Induction\/} holds in this new setting, prove soundness of a Hoare calculus relative to our semantics, give a direct calculus ${\cal C}$ on programs, and prove that the denotation of any program $P$ in our semantics is equal to the union of the denotations of all those programs $L$ such that $P$ follows from $L$ in our calculus and $L$ does not contain recursion or choice.

But What Have You Done for Us Lately?: Some Recent Perspectives on Linguistic Nativism

2001-02-23Z

The problem with many contemporary criticisms of Chomsky and linguistic nativism is that they are based upon features of the theory that are no longer germane; aspects that have either been superseded by more adequate proposals, or that have been dropped altogether under the weight of contravening evidence. In this paper, rather than rehashing old debates that are voluminously documented elsewhere, we intend to focus on more recent developments. To this end, we have put a premium on references from the 1990s and the latter half of the 1980s. First, we will describe exactly what is now thought to be innate about language, and why it is thought to be innate rather than learned. Second, we will examine the evidence that many people take to be the greatest challenge to the nativist claim: ape language. Third, we will briefly consider how an innate language organ might have evolved. Fourth we will look at how an organism might communicate without benefit of the innate language structure proposed by Chomsky, and examine a number of cases in which this seems to be happening. Finally we will try to sum up our claims and characterize what we believe will be the most fruitful course of debate for the immediate future.

Constructing semantic representations using the MDL principle

2001-05-08Z

Words receive a significant part of their meaning from use in communicative settings. The formal mechanisms of lexical acquisition, as they apply to rich situational settings,may also be studied in the limited case of corpora of written texts. This work constitutes an approach to deriving semantic representations for lexemes using techniques from statistical induction. In particular, a number of variations on the MDL principle were applied to selected sample sets and their influence on emerging theories of word meaning explored. We found that by changing the definition of description length for data and theory - which is equivalent to different encodings of data and theory - we may customize the emerging theory, augmenting and altering frequency effects. Also the influence of the stochastic properties of the size of the theory has been demonstrated. The results consist in a set of distributional properties of lexemes, which reflect cognitive distinctions in the meaning of words.

Context as a Social Construct

1998-06-24Z

This position paper argues that in addition to the familiar approach using formal contexts, there is now a need in AI to study contexts as social constructs. As a successful example of the latter approach, I draw attention to `interpretation' (in the sense of literary theory), viz. the reconstruction of intended meaning of a literary text that takes into account the context in which the author assumed the reader would place the text. An important contribution here comes from Harris (1988), enumerating the seven crucial dimensions of context: knowledge of reality, knowledge of language, and the authorial, generic, collective, specific, and textual dimensions. Finally, two thought-provoking papers in interpretation, (Barwise 1989) and (Hobbs 1990), are analyzed as useful attempts which also come to grips with the notion of context.

Current Approaches to Punctuation in Computational Linguistics

1998-06-16Z

Some recent studies in computational linguistics have aimed to take advantage of various cues presented by punctuation marks. This short survey is intended to summarise these research efforts and additionally, to outline a current perspective for the usage and functions of punctuation marks. We conclude by presenting an information-based framework for punctuation, influenced by treatments of several related phenomena in computational linguistics.

Dyslexic and Category-Specific Impairments in a Self-Organizing Feature Map Model of the Lexicon

1999-01-03Z

DISLEX is an artificial neural network model of the mental lexicon. It was built to test computationally whether the lexicon could consist of separate feature maps for the different lexical modalities and the lexical semantics, connected with ordered pathways. In the model, the orthographic, phonological, and semantic feature maps and the associations between them are formed in an unsupervised process, based on cooccurrence of the lexical symbol and its meaning. After the model is organized, various damage to the lexical system can be simulated, resulting in dyslexic and category-specific aphasic impairments similar to those observed in human patients.

Extraction of Keyphrases from Text: Evaluation of Four Algorithms

2001-09-17Z

This report presents an empirical evaluation of four algorithms for automatically extracting keywords and keyphrases from documents. The four algorithms are compared using five different collections of documents. For each document, we have a target set of keyphrases, which were generated by hand. The target keyphrases were generated for human readers; they were not tailored for any of the four keyphrase extraction algorithms. Each of the algorithms was evaluated by the degree to which the algorithms keyphrases matched the manually generated keyphrases. The four algorithms were (1) the AutoSummarize feature in Microsofts Word 97, (2) an algorithm based on Eric Brills part-of-speech tagger, (3) the Summarize feature in Veritys Search 97, and (4) NRCs Extractor algorithm. For all five document collections, NRCs Extractor yields the best match with the manually generated keyphrases.

The Many Functions of Discourse Particles: A Computational Model of Pragmatic Interpretation

2011-12-16T00:11:43Z

We present a connectionist model for the interpretation of discourse particles in real dialogues that is based on neuronal principles of categorization (categorical perception, prototype formation, contextual interpretation). It can be shown that discourse particles operate just like other morphological and lexical items with respect to interpretation processes. The description proposed locates discourse particles in an elaborate model of communication which incorporates many different aspects of the communicative situation. We therefore also attempt to explore the content of the category discourse particle. We present a detailed analysis of the meaning assignment problem and show that 80%– 90% correctness for unseen discourse particles can be reached with the feature analysis provided. Furthermore, we show that ‘analogical transfer’ from one discourse particle to another is facilitated if prototypes are computed and used as the basis for generalization. We conclude that the interpretation processes which are a part of the human cognitive system are very similar with respect to different linguistic items. However, the analysis of discourse particles shows clearly that any explanatory theory of language needs to incorporate a theory of communication processes.

A Simple-Minded Network Model with Context-like Objects

1998-08-07Z

The aim of this paper is to describe a simple extension of semantic nets. In this formulation we have labelled nodes with directed arcs, but the directed arcs can lead to other arcs as well as nodes. In this model contexts are not differentiated as special objects, but rather that some nodes to a greater or lesser extent have roles as encoders of contextual information. This formulation is shown to be expressive enough to capture several aspects of context, namely: context-dependent inference, context specific learning, the selection of a relevant context and the generalisation of knowledge. Its strengths are its simplicity, the fact that it can relate and integrate several aspects of context and its connections with formal logic. It is not claimed that this is a model of any type of context found in human activity.

Situated Nonmonotonic Temporal Reasoning with BABY-SIT

1998-06-16Z

After a review of situation theory and previous attempts at `computational' situation theory, we present a new programming environment, BABY-SIT, which is based on situation theory. We then demonstrate how problems requiring formal temporal reasoning can be solved in this framework. Specifically, the Yale Shooting Problem, which is commonly regarded as a canonical problem for nonmonotonic temporal reasoning, is implemented in BABY-SIT using Yoav Shoham's causal theories.

The Dilemma of Saussurean Communication

1998-06-15Z

A Saussurean communication system exists when an entire communicating population uses a single "language" that maps states unambiguously onto symbols and then back into the original states. This paper describes a number of simulations performed with a genetic algorithm to investigate the conditions necessary for such communication systems to evolve. The first simulation shows that Saussurean communication evolves in the simple case where direct selective pressure is placed on individuals to be both good transmitters and good receivers. The second simulation demonstrates that, in the more realistic case where selective pressure is only placed on doing well as a receiver, Saussurean communication fails to evolve. Two methods, inspired by research on the Prisoner's Dilemma, are used to attempt to solve this problem. The third simulation shows that, even in the absence of selective pressure on transmission, Saussurean communication can evolve if individuals interact multiple times with the same communication partner and are given the ability to respond differentially based on past interaction. In the fourth simulation, spatially organized populations are used, and it is shown that this allows Saussurean communication to evolve through kin selection.

Information-Oriented Computation with BABY-SIT

1998-06-19Z

While situation theory and situation semantics provide an appropriate framework for a realistic model-theoretic treatment of natural language, serious thinking on their `computational' aspects has only recently started. Existing proposals mainly offer a Prolog- or Lisp-like programming environment with varying degrees of divergence from the ontology of situation theory. In this paper, we introduce a computational medium (called BABY-SIT) based on situations. The primary motivation underlying BABY-SIT is to facilitate the development and testing of programs in domains ranging from linguistics to artificial intelligence in a unified framework built upon situation-theoretic constructs.

Language polygenesis: A probabilistic model

1998-06-24Z

Monogenesis of language is widely accepted, but the conventional argument seems to be mistaken; a simple probabilistic model shows that polygenesis is likely. Other prehistoric inventions are discussed, as are problems in tracing linguistic lineages. Language is a system of representations; within such a system, words can evoke complex and systematic responses. Along with its social functions, language is important to humans as a mental instrument. Indeed, the invention of language,that is the accumulation of symbols to represent emotions, objects, and acts may be the most important event in human evolution, because so many developments follow from it. For example, Edward Sapir speculated that some embryonic form of language must have been available to early man to help him fashion tools from stone (Sapir,1921). Sophisticated biface stone tools date to early Homo erectus some 1.5 million years ago, suggesting a similar age for language. This paper considers whether the invention of language occurred at only one pre-historic site or at several sites. In other words, did language emerge by monogenesis or polygenesis? Early thinkers believed in monogenesis, against a background of divine creation. Perhaps the best known account is the biblical story of Adam giving names to plants and animals in the Garden of Eden. Similar legends are found among many peoples. Modern linguists too assume monogenesis, but on probabilistic grounds (see, for instance, Southworth and Daswani, 1974, p.314). The argument seems to be that the invention of language is an extremely unlikely event, because symbolization involves abstraction and requires synchronized insight by several individuals; therefore, the probability of occurrence at more than one site must be vanishingly small. We have found no explicit quantitative treatment of this question in the literature, but the underlying logic has to be the multiplication of probabilities. If p is small at one site,then p.p for two sites is smaller still, and so on. This reasoning is false, as we show here. The fallacy lies in the focus on two particular sites rather than consideration of all pairs of sites.

On the Virtues of Theft Over Honest Toil: Grounding Language and Thought in Sensorimotor Categories

2001-06-19Z

What language allows us to do is to steal categories through hearsay instead of having to earn them through the honest toil of learning them from direct exposure and trial and error feedback from the consequences of miscategorisation. To make us capable of theft, however, the content-bearing symbols of language must ultimately be grounded in categories that have been earned through honest toil (unless "prepared" by Darwinian theft); it cannot be linguistic theft all the way down. Category names must be grounded in the capacity to sort, label and interact with the proximal sensorimotor projections of their distal members in a way which coheres systematically with their interpretations, singly, and strung together to express propositions.

Steps toward Formalizing Context

1998-06-17Z

The importance of contextual reasoning is emphasized by various researchers in AI. (A partial list includes John McCarthy and his group, R. V. Guha, Yoav Shoham, Giuseppe Attardi and Maria Simi, and Fausto Giunchiglia and his group.) Here, we survey the problem of formalizing context and explore what is needed for an acceptable account of this abstract notion.

Subsymbolic Case-Role Analysis of Sentences with Embedded Clauses

2001-11-18Z

A distributed neural network model called SPEC for processing sentences with recursive relative clauses is described. The model is based on separating the tasks of segmenting the input word sequence into clauses, forming the case-role representations, and keeping track of the recursive embeddings into different modules. The system needs to be trained only with the basic sentence constructs, and it generalizes not only to new instances of familiar relative clause structures, but to novel structures as well. SPEC exhibits plausible memory degradation as the depth of the center embeddings increases, its memory is primed by earlier constituents, and its performance is aided by semantic constraints between the constituents. The ability to process structure is largely due to a central executive network that monitors and controls the execution of the entire system. This way, in contrast to earlier subsymbolic systems, parsing is modeled as a controlled high-level process rather than one based on automatic reflex responses.

Book Review -- Vladimir Lifschitz, ed., Formalizing Common Sense: Papers by John McCarthy

1998-06-26Z

This is a review of Formalizing Common Sense: Papers by John McCarthy, ed. by Vladimir Lifschitz, published by Ablex Publishing Corp. in 1990.

Book Review -- Hans Kamp and Uwe Reyle, From Discourse to Logic: Introduction to Model-theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory

1998-06-26Z

This is a review of From Discourse to Logic: Introduction to Model-theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory, by Hans Kamp and Uwe Reyle, published by Kluwer Academic Publishers in 1993.

Situated Modeling of Epistemic Puzzles

1998-06-19Z

Situation theory is a mathematical theory of meaning introduced by Jon Barwise and John Perry. It has evoked great theoretical interest and motivated the framework of a few `computational' systems. PROSIT is the pioneering work in this direction. Unfortunately, there is a lack of real-life applications on these systems and this study is a preliminary attempt to remedy this deficiency. Here, we solve a group of epistemic puzzles using the constructs provided by PROSIT.

Towards Situation-Oriented Programming Languages

1998-06-19Z

Recently, there have been some attempts towards developing programming languages based on situation theory. These languages employ situation-theoretic constructs with varying degrees of divergence from the ontology of the theory. In this paper, we review three of these programming languages.

Applying semantic concepts to the media assigment problem in multi-media communication.

1999-08-05Z

Our long term goal is an understanding of human communication in terms which would provide the basis for rational design. The kernel would be a theory of the cognitive consequences of allocating the same information to different media and modalities, based on the user's information processing characterised in computational terms. Our theory of the cognitive consequences of media/modality allocation starts from an analysis of differences in logical expressiveness of graphical and linguistic representations (Stenning \& Oberlander (1994, 1995)). This semantic approach requires conceptualisations of {\it medium} and {\it modality} that can be related to representation systems. We propose that media are the physical/perceptual aspects of representations; modalities are classes of interpretation function which map media onto meanings. These interpretations of the terms contrast with existing HCI usage. The curtailments of abstraction observed in graphics arise from interactions between medium and modality. Graphical modalities are distinguished from sentential languages by the nature of their interpretation functions. A hierarchy of expressiveness of interpretations of graphics is defined, and compared with interpretations of sentential languages. Using the expressiveness of representations to predict their cognitive properties also requires reference to the availability of constraints of their interpretation to users. Further contrasts between graphics and language emerge in the availability of their constraints. Three example domains of graphical representations are analysed from this perspective---matrix graphics; logic diagrams; and semantic networks. Some empirical evidence of the usability of these notations is reviewed as evidence that the proposed conceptualisation offers powerful generalisations.

Automated Understanding of Financial Statements Using Neural Networks and Semantic Grammars

2003-04-26Z

This article discusses how neural networks and semantic grammars may be used to locate and understand financial statements embedded in news stories received from on-line news wires. A neural net is used to identify where in the news story a financial statement appears to begin. A grammar then is applied to this text in an effort to extract specific facts from the financial statement. Applying grammars to financial statements presents unique parsing problems since the dollar amounts of financial statements are typically arranged in multiple columns, with small paragraphs of text above each column. Text therefore is meant to be read both vertically and horizontally, in contrast to ordinary news text, which is read only horizontally.

Contexts, Oracles, and Relevance

1998-06-24Z

We focus on how we should define the relevance of information to a context for information processing agents, such as oracles. We build our formalization of relevance upon works in pragmatics which refer to contextual information without giving any explicit representation of context. We use a formalization of context (due to us) in Situation Theory, and demonstrate its power in this task. We also discuss some computational aspects of this formalization.

Modeling Context with Situations

1998-06-24Z

The issue of context arises in assorted areas of Artificial Intelligence. Although its importance is realized by various researchers, there is not much work towards a useful formalization. In this paper, we will present a preliminary model (based on Situation Theory) and give examples to show the use of context in various fields, and the advantages gained by the acceptance of our proposal.

Situations and Computation: An Overview of Recent Research

1998-06-24Z

Serious thinking about the computational aspects of situation theory is just starting. There have been some recent proposals in this direction (viz. PROSIT and ASTL), with varying degrees of divergence from the ontology of the theory. We believe that a programming environment incorporating bona fide situation-theoretic constructs is needed and describe our very recent BABY-SIT implementation. A detailed critical account of PROSIT and ASTL is also offered in order to compare our system with these pioneering and influential frameworks.

Understanding Radio Broadcasts On Soccer: The Concept `Mental Image' and Its Use in Spatial Reasoning

2001-05-29Z

Most cognitive theories agree that a listener of a sports broadcast on radio usually imagines the scene described; the concept `mental image' appears in a specific sort of explanations. In contrast to this conception, it is argued that this concept should rather be understood as part of a certain kind of grounding explanations of the radio listener's understanding. This particular conception is based on the distinction between `specification' and `implementation' as found in the theory of abstract data types. Its application to the field of spatial concepts leads to a computational system (ANTLIMA) which exemplifies how the expression `mental image' could be used while explaining a speaker's ability to control the resolvability of ambiguities in an objective report of what the speaker sees.

Computational Situation Theory

1998-06-19Z

Situation theory has been developed over the last decade and various versions of the theory have been applied to a number of linguistic issues. However, not much work has been done in regard to its computational aspects. In this paper, we review the existing approaches towards `computational situation theory' with considerable emphasis on our own research.

Lexical Disambiguation Based on Distributed Representations of Context Frequency

2002-12-11Z

A model for lexical disambiguation is presented that is based on combining the frequencies of past contexts of ambiguous words. The frequencies are encoded in the word representations and define the words' semantics. A Simple Recurrent Network (SRN) parser combines the context frequencies one word at a time, always producing the most likely interpretation of the current sentence at its output. This disambiguation process is most striking when the interpretation involves semantic flipping, that is, an alternation between two opposing meanings as more words are read in. The sense of ``throwing a ball'' alternates between ``dance'' and ``baseball'' as indicators such as the agent, location, and recipient are input. The SRN parser demonstrates how the context frequencies are dynamically combined to determine the interpretation of such sentences. We hypothesize that several other aspects of ambiguity resolution are based on similar mechanisms, and can be naturally approached from the distributed connectionist viewpoint.

Situated Processing of Pronominal Anaphora

1998-06-23Z

We describe a novel approach to the analysis of pronominal anaphora in Turkish. A computational medium which is based on situation theory is used as our implementation tool. The task of resolving pronominal anaphora is demonstrated in this environment which employs situation-theoretic constructs for processing.

Book Review -- Peter D. Mosses, Action Semantics

1998-07-02Z

This is a review of Action Semantics, by Peter D. Mosses, published by Cambridge University Press in 1992.

ANTLIMA -- A Listener Model with Mental Images

2001-05-29Z

AI research concerning the connection between seeing and speaking mainly employs what is often called reference semantics. Applying this approach to the situation of a radio sports reporter, we have to coordinate the demand of referentially anchoring an utterance dealing with the visually perceived, and the demand for coherence of an utterance as part of a verbal interaction with somebody not situated in the same perceptual context. In consequence, we are led to the conception of a speaker anticipating the listeners' understanding by means of mental images which replace the percepts being described, and thus provide the referents for the audience. We present a system realizing this type of partner modeling, emphasizing mainly the reconstruction of the referents, i.e., of a mental image. Starting from the thesis that the audience expects the speaker to mean the most typical case of the described class of events or situations with respect to the communicated context, we explain a mechanism for representing and using typicality distributions of static spatial relations which is related to Herskovits' analytical framework. Extended to restrictions of speed and temporal duration, this mechanism also allows us to construct dynamic mental images corresponding to the referents of objective sports reports.

BABY-SIT: A Computational Medium Based on Situations

1998-06-23Z

While situation theory and situation semantics provide an appropriate framework for a realistic model-theoretic treatment of natural language, serious thinking on their `computational' aspects has just started. Existing proposals mainly offer a Prolog- or Lisp-like programming environment with varying degrees of divergence from the ontology of situation theory. In this paper, we introduce a computational medium (called BABY-SIT) based on situations. The primary motivation underlying BABY-SIT is to facilitate the development and testing of programs in domains ranging from linguistics to artificial intelligence in a unified framework built upon situation-theoretic constructs.

Natural Language Processing with Modular Neural Networks and Distributed Lexicon

2001-11-18Z

An approach to connectionist natural language processing is proposed, which is based on hierarchically organized modular Parallel Distributed Processing (PDP) networks and a central lexicon of distributed input/output representations. The modules communicate using these representations, which are global and publicly available in the system. The representations are developed automatically by all networks while they are learning their processing tasks. The resulting representations reflect the regularities in the subtasks, which facilitates robust processing in the face of noise and damage, supports improved generalization, and provides expectations about possible contexts. The lexicon can be extended by cloning new instances of the items, that is, by generating a number of items with known processing properties and distinct identities. This technique combinatorially increases the processing power of the system. The recurrent FGREP module, together with a central lexicon, is used as a basic building block in modeling higher level natural language tasks. A single module is used to form case-role representations of sentences from word-by-word sequential natural language input. A hierarchical organization of four recurrent FGREP modules (the DISPAR system) is trained to produce fully expanded paraphrases of script-based stories, where unmentioned events and role fillers are inferred.

Elephant 2000 - A programming language based on speech acts

1998-02-27Z

Elephant 2000 is a proposed programming language good for writing and verifying programs that interact with people (eg. transaction processing) or interact with programs belonging to other organizations (eg. electronic data interchange) 1. Communication inputs and outputs are in an I-O language whose sentences are meaningful speech acts identified in the language as questions, answers, offers, acceptances, declinations, requests, permissions and promises. 2. The correctness of programs is partly defined in terms of proper performance of the speech acts. Answers should be truthful and responsive, and promises should be kept. Sentences of logic expressing these forms of correctness can be generated automatically from the form of the program. 3. Elephant aource programs may not need data structures, because they can refer directly to the past. Thus a program can say that an airline passenger has a reservation if he has made one and hasn't cancelled it. 4. Elephant programs themselves can be represented as sentences of logic. Their extensional properties follow from this representation without an intervening theory of programming or anything like Hoare axioms. 5. Elephant programs that interact non-trivially with the outside world can have both input-output specification, relating the programs inputs and outputs, and accomplishment specifications concerning what the program accomplishes in the world. These concepts are respectively generalizations of the philosophers' illocutionary and perlocutionary speech acts. 6. Programs that engage in commercial transactions assume obligations on behalf of their owners in exchange for obligations assumed by other entities. It may be part of the specification of an Elephant 2000 program that these obligations are exchanged as intended, and this too can be expressed by a logical sentence. 7. Human speech acts involve intelligence. Elephant 2000 is on the borderline of AI, but the article emphasizes the Elephant usages that do not require AI.

Summary in English of PRINCIPES DES SYSTEMES INTELLIGENTS

1998-06-29Z

This book, published in a series called Cognitive Science, aims -- as suggested in the title -- at defining general properties of « intelligent systems ». The question of the intelligence of a computer system running a piece of software can be considerably simplified if no consideration is given to the concept of a « subject » displaying intelligent behavior. In everyday life we content ourselves with simply assuming that our human interlocutors are inhabited by a « subject »: we never demand any proof of the actual existence of such a subject. Searles « Chinese room » thought experiment shows a human subject being fluent in Chinese but being unaware of such capacity for being solely aware of manipulating skillfully sets of symbols. For all practical purpose all persons dealing with the prisoner of the Chinese room would be justified to regard themselves as dealing with a fluent speaker of Chinese. In the circumstance the « subjective » feeling of the subject that he understands or not Chinese is irrelevant to his actual fluency. The same would apply to a computer system: as long as the sentences it produces are indistinguishable from those that a human being would generate, it would justifiably be regarded as intelligent. The question reduces then to that of designing a function and data structure that generates algorithmically specific sentences by traveling through the word-space of a particular languages lexicon. The first constraint to be observed is that of grammatical acceptability of the clauses generated. The second is that of the plausibility of these sentences within a general context of human cultures (common sense view). The third constraint is that of topicality. The fourth constraint is that of non-contradiction between consecutive clauses. Little by little a picture emerges of a network of memory traces capable of growing in an organized manner associated with an affect dynamics capable of generating from these memory traces, clauses displaying emergent logical features.

Rethinking the language bottleneck: Why don't animals learn to communicate?

1998-06-15Z

While most work on the evolution of language has been centered on the evolution of syntax, my focus in this paper is instead on more basic features that separate human communication from the systems of communication used by other animals. In particular, I argue that human language is the only existing system of learned arbitrary reference. While innate communication systems are, by definition, directly transmitted genetically, the transmission of a learned learned systems must be indirect. Learners must acquire the system by being exposed its the use in the community. Although it is reasonable that a learner has access to the utterances that are produced, it is less clear how accessible the meaning is that the utterance is intended to convey. This particularly problematic if the system of communication is symbolic -- where form and meaning are linked in a purely conventional way. Given this, I propose that the ability to transmit a learned symbolic system of communication from one generation to the next represents a key milestone in the evolution of language.

The learning barrier: Moving from innate to learned systems of communication

1998-06-15Z

Human language is a unique ability. It sits apart from other systems of communication in two striking ways: it is syntactic, and it is learned. While most approaches to the evolution of language have focused on the evolution of syntax, this paper explores the computational issues that arise in shifting from a simple innate communication system to an equally simple one that is learned. Associative network learning within an observational learning paradigm is used to explore the computational difficulties involved in establishing and maintaining a simple learned communication system. Because Hebbian learning is found to be sufficient for this task, it is proposed that the basic computational demands of learning are unlikely to account for the rarity of even simple learned communication systems. Instead, it is the problem of *observing* that is likely to be central -- in particular the problem of determining what meaning a signal is intended to convey.

A Holistic Approach to Language

1999-03-16Z

The following progress report views language acquisition as primarily the attempt to create processes that connect together in a fruitful way linguistic input and other activity. The representations made of linguistic input are thus those that are optimally effective in mediating such interconnections. An effective Language Acquisition Device should contain mechanisms specific to the task of creating the desired interconnection processes in the linguistic environment in which the language learner finds himself or herself. Analysis of this requirement gives clear indications as to what these mechanisms may be.

Computing Machinery and Intelligence

1998-07-24Z

I propose to consider the question, "Can machines think?" This should begin with definitions of the meaning of the terms "machine" and "think." The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous, If the meaning of the words "machine" and "think" are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, "Can machines think?" is to be sought in a statistical survey such as a Gallup poll. But this is absurd. Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words. The new form of the problem can be described in terms of a game which we call the 'imitation game." It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart front the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either "X is A and Y is B" or "X is B and Y is A." The interrogator is allowed to put questions to A and B. We now ask the question, "What will happen when a machine takes the part of A in this game?" Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, "Can machines think?"