 |
|
|
 |
|
|
|
|
Stubb |
| | | | | | posted 5/10/2010 19:26 |   | | | | | | | | | I've become interested in AI here lately, but will be the first to admit It's definitely difficult to comprehend.
I mean, I've seen responses on different topics that suggest a completely 'non hard-coded' application. But there has to be atleast a basic knowledge base to learn from right? Like simple mathmatics, simple responses and being able to realize if it knows something or not (database).
I've been looking into sentence interpreting, and it's almost like there is just a big collection of responses for certain input. So it really isn't AI, it's just the intelligence of the developers.
I've started to work on a simple application that can distinguish between a question or a statement and if there is no puncuation, the problem becomes quite difficult.
Are programs like HAL and ALAN just a bunch of if/else conditionals?
Stubb
| | |
| | |
hunt |
| | | | | | posted 5/11/2010 02:53 | | | | | | | | | | There are others on this forum that can explain the inner workings of HAL and ALAN much better than I, however I'd like to comment on the problem of recognizing questions.
I'm working on a parser myself, and this is a much more difficult challenge than one initially suspects. There are certain string patterns that characterize a question. And then there is the added factor of context (is the user likely to be asking a question given the topic of discussion, what we have just been saying, what words are included in what was just said, etc.).
Being able to account for all of this requires exposure and active teaching. Certainly hard coding can speed this process along, especially when it comes to recognizing common string patterns. But then, one can argue that a more robust system should be able to learn new string patterns based on exposure to natural language. (Not just in a teaching environment, but by using non-teaching focused encounters to learn based on whether or not the user responds well to the string interpretation that the AI guesses.)
It's all a hairy mess. But I'm poking at it. Please post more as your project progresses!
| | |
| | |
Stubb |
| | | | | | posted 5/11/2010 16:28 | | | | | | | | | | | |
"I'm working on a parser myself, and this is a much more difficult challenge than one initially suspects. There are certain string patterns that characterize a question. And then there is the added factor of context (is the user likely to be asking a question given the topic of discussion, what we have just been saying, what words are included in what was just said, etc.). "
Yeah the intial decision of whether or not the sentence is a question (simple question) wasn't that bad. As soon as you want to know the context though, that's where it get's hairy.
"Being able to account for all of this requires exposure and active teaching. Certainly hard coding can speed this process along, especially when it comes to recognizing common string patterns. But then, one can argue that a more robust system should be able to learn new string patterns based on exposure to natural language. (Not just in a teaching environment, but by using non-teaching focused encounters to learn based on whether or not the user responds well to the string interpretation that the AI guesses.) "
I just don't understand how you could create a system that can learn without having a basic knowledge base to build upon.
"It's all a hairy mess. But I'm poking at it. Please post more as your project progresses! "
It might be awhile before I get anything worth posting about : ) Good luck!
| |
I put it all in quote : (.
Stubb
| | |
| | |
Stubb |
| | | | | | posted 5/11/2010 16:34 | | | | | | | | | | I'm messing around with this stuff in C#.NET, would you suggest a different programming language or is .net find for novice applications?
I have heard of Lisp in my "Survey of programming languages" course in college, but I thought it was just some old lang. that USED to be used to program AI.
Thanks again,
Stubb
| | |
| | |
tkorrovi |
| | | | | | posted 5/11/2010 16:47 | | | | | | | | | | I must admit that i shiver when hearing about .net and java. But anyway, both are interpreted languages (byte code, well, most of the interpreted languages have byte code) and thus they for the very theoretical reasons *must* be slow, no matter what some may say, whether something is fast enough is often subjective, anyway. But beyond doubt they are slow for an advanced AI. Which doesn't mean that they don't do fast things in java, i have seen even 3d applications in java, and they were horribly slow.
Yes C is the fastest, fortran is even faster, but mostly for restricted range of tasks in mathematics.
But sure they teach only java in most universities and such. I hope that the popularity of java would decrease in the future, which should happen when the software development follows any logic.
| | | Last edited by tkorrovi @ 5/11/2010 4:49:00 PM | | |
| | |
Stubb |
| | | | | | posted 5/11/2010 17:13 | | | | | | | | | | "Yes C is the fastest, fortran is even faster, but mostly for restricted range of tasks in mathematics."
I might start programming in C once I get my application actually doing something significant, but wouldn't mind learning Lisp or prolog either.
"But sure they teach only java in most universities and such. I hope that the popularity of java would decrease in the future, which should happen when the software development follows any logic."
I've noticed this as well. The strange thing is that I got my CS degree in Texas and there aren't many jobs in the texas area that use Java, .NET is pretty much the dominant language. I've heard java is really big on the west coast though. They told us, "We want to teach you what is being used a lot in industry, so you can get a job fast(er)", but I would have to say that around here, Java will get you no where pretty fast.
Thanks for the advice,
Stubb
| | |
| | |
Stubb |
| | | | | | posted 5/11/2010 20:37 | | | | | | | | | | For an application to be able to interpret any sentence directed to it in conversation, wouldn't that application have to already be able to understand the symantecs of the given natural language? Atleast an elementary collection of knowledge? And if it was an incomplete knowledge, be able to question what certain words are and then store the definitions/descriptions and contexts in which that word can be used.
Even if the goal is to start with a blank slate, how would the application know to question what something means? Even if it did have a rudamentary collection of words that it could use to compare a given sentence, those words would have to be categorized so the application can analyze the sentence properly. e.g. Knowing what words can be used as Adjectives, Adverbs, Verbs, Nouns and etc. Knowing the basic and eventually the complex structures of sentences in that language.
I've just been thinking about it a lot and for me, for an application to "learn" it has to be taught how to learn and have a base knowledge pool to build upon.
But even after you figure all this out the next objective would be for the application to be able to take a collection of words from it's vocabulary and create a response that is coherent and in context with the ongoing conversation. I just thought the first part was difficult : (.
Stubb
| | |
| | |
hunt |
| | | | | | posted 5/11/2010 21:45 | | | | | | | | | | | |
Stubb wrote @ 5/11/2010 8:37:00 PM:
For an application to be able to interpret any sentence directed to it in conversation, wouldn't that application have to already be able to understand the symantecs of the given natural language? Atleast an elementary collection of knowledge? And if it was an incomplete knowledge, be able to question what certain words are and then store the definitions/descriptions and contexts in which that word can be used.
Even if the goal is to start with a blank slate, how would the application know to question what something means? Even if it did have a rudamentary collection of words that it could use to compare a given sentence, those words would have to be categorized so the application can analyze the sentence properly. e.g. Knowing what words can be used as Adjectives, Adverbs, Verbs, Nouns and etc. Knowing the basic and eventually the complex structures of sentences in that language.
I've just been thinking about it a lot and for me, for an application to "learn" it has to be taught how to learn and have a base knowledge pool to build upon.
But even after you figure all this out the next objective would be for the application to be able to take a collection of words from it's vocabulary and create a response that is coherent and in context with the ongoing conversation. I just thought the first part was difficult : (.
Stubb
| |
whoops, double post. see below...
| | | Last edited by hunt @ 5/11/2010 9:46:00 PM | | |
| | |
hunt |
| | | | | | posted 5/11/2010 21:46 | | | | | | | | | | I started my project using C++ for the simple reason that it was the language I was most familiar with at the time. I quickly realized that C++ hates strings. (At least, this was my experience, your mileage may vary.) So I switched over to python and I love it. Now, python is slow slow slow. However, it is great with strings.
"For an application to be able to interpret any sentence directed to it in conversation, wouldn't that application have to already be able to understand the symantecs of the given natural language? Atleast an elementary collection of knowledge? [...]"
The parser I am developing accesses the WordNet dictionary via the Natural Language Toolkit. This toolkit allows the parser to determine, roughly, whether a word is a noun, verb, etc. I've written my own toolkits to handle verb conjugation and plural/singular noun forms.
However, language is ambiguous as we all well know. The parser also searches through string patterns and grammar patterns that it has encountered before to try to optimize the parse. This is the primary sentence learning mechanism.
"But even after you figure all this out the next objective would be for the application to be able to take a collection of words from it's vocabulary and create a response that is coherent and in context with the ongoing conversation. I just thought the first part was difficult : (."
In my program, after parsing, each sentence is broken down into a tree, with the sentence subject at the base, and the predicate as a branch on that tree. This is the way that factual knowledge is stored. Each "fact" (subject, verb, additional conditions) has an id tag that other "facts" can reference as conditions.
I'm hopeful that this storage mechanism will be beneficial when I write functions that handle searching for question answers, combining facts into chronologies or "stories", and attempting to give conversations context.
I think the real fail-point is breaking down parsing and contextualizing into separate tasks. I'm beginning to realize that my parsing will become much more robust once it is able to search the conversation context. I've got a lot of ideas (algorithms) in this direction, but I want to perfect the fact storage mechanism first.
| | |
| | |
Stubb |
| | | | | | posted 5/11/2010 22:12 | | | | | | | | | | "The parser I am developing accesses the WordNet dictionary via the Natural Language Toolkit. This toolkit allows the parser to determine, roughly, whether a word is a noun, verb, etc. I've written my own toolkits to handle verb conjugation and plural/singular noun forms."
How complex of sentences are you able to parse right now?
"However, language is ambiguous as we all well know. The parser also searches through string patterns and grammar patterns that it has encountered before to try to optimize the parse. This is the primary sentence learning mechanism."
How does the parser know what to consider a 'valid pattern' in a given string and then store it for later comparisons?
Sorry if im too 'questiony'
Stubb
| | |
| | |
hunt |
| | | | | | posted 5/11/2010 23:13 | | | | | | | | | | "How complex of sentences are you able to parse right now?"
It varies widely. Most sentences that begin with the subject and only have clauses and dependent phrases at the end can be parsed with the first stage of the parser. This stage includes certain hard-wired methods for determining how likely one parse is over another. (Are all articles followed by nouns? Is there a verb in the sentence, if the sentence is a certain length? etc.)
More complex sentences are understood if something similar has been encountered before. The patterns that it matches use easily-parsed words (prepositions, articles, and certain other words) to delineate the pattern, and it makes allowances for the fact that different instances of the pattern might use a different number of adjectives, adverbs, and other filler.
Patterns are then rated on how well they match the input sentence, given what is known or guessed about the grammar of the input sentence. The rating scheme is more art than science, and I tweak it as I go.
"How does the parser know what to consider a 'valid pattern' in a given string and then store it for later comparisons?"
Barring one chat mode that was built to test the knowledge base (a very very simple chat mode), every form of communication with my program is a type of learning mode. Most of these include the option to turn on "grammar" learning as well. Thus far, new patterns have only been learned through these modes so I can monitor them for correctness. However, everything is in place for the program to store patterns with a confidence rating (starting low, then increasing for each time the pattern in encountered).
In theory, whenever a pattern is used to interpret user input in conversation, and the response is well-received by the user, then confidence in the pattern should increase. Likewise, when reading text, if the same word is parsed differently in sentences about the same topic (sentences occurring in the same paragraph or lesson file, etc.) then chances are something is wrong with one of the patterns used to parse, and confidence goes down.
Determining how to increment confidence I think will be a matter of experience and tweaking.
"Sorry if im too 'questiony'"
These are great questions, and isn't this sort of discussion what these forums are for? :)
| | |
|
|
|
|
|
