Quote:
Originally posted by G_yoshi
Eh?
I just fail to see the reason to duplicate an existing command. If it fails to work properly, then post it in the bugs forum  Maybe Stefan will go back and make sure it works for the next release 
|
tokenize2 +,bob+went to+have+sex;
#t(0)=bob
#t(1)=went
#t(2)=to
#t(3)=have
#t(4)=sex
MY way
tokenize3 +,bob+went to+have+sex;
#t(0)=bob
#t(1)=went to
#t(2)=have
#t(3)=sex
tokenize3 wouldnt do the ADDITIONAL delimeters it would just simply use new delimeters and remove all old forms.
no spaces etc...