
06-06-2001, 07:25 AM
|
|
Registered User
|
Join Date: May 2001
Location: Olympia, Washington USA
Posts: 25
|
|
Also...
|
Tokenize breaks up a string into sections, or tokens. Take this string for example:
Tokenize this string for me
If you call tokenize with this string, it would create:
#t(0)=tokenize
#t(1)=this
#t(2)=string
#t(3)=for
#t(4)=me
There are some exceptions:
Tokenize "this string" for me
When tokenized:
#t(0)=tokenize
#t(1)=this string
#t(2)=for
#t(3)=me
You can see that everything inbetween parenthesis are taken as 1 token. There you go. |
|
|
|