tokenize3

[Warcaptain]

there should be a tokenize3 command that tokenizes without tokenizing with spaces.


ie: tokenize3 |,bob|went to the|market;

token1=bob
token2=went to the
token3=market

[jeff335]

Er, future improvements?



But it sounds good.

[zorakid2001]

WTF?? i dont understand

[zorakid2001]

oh now i do sorry for being a retard

[Python523]

you could just have the string be like "bob" "went to" "the market"
0 - bob
1 - went to
2 - the market

[Tyhm]

Except tokenize "bob" "went to" "the market" returns
"bob"
"went
to"
"the
market"
is his point

[Python523]

Originally posted by Tyhm
Except tokenize "bob" "went to" "the market" returns
"bob"
"went
to"
"the
market"
is his point

Last I checked when you tokened something in quotes, it took everything inside the quotes as 1 token, I'll check again just to make sure later =/

[adam]

Originally posted by Python523

Last I checked when you tokened something in quotes, it took everything inside the quotes as 1 token, I'll check again just to make sure later =/

I think so too.

[screen_name]

eek

i have this script



if (actionserverside) {
with (getplayer(#p(0))) {
setstring client.test,#p(1);
}
}
//#CLIENTSIDE
if (playerchats && startswith(-,#c)) {
triggeractoin 0,0,serverside,test,"#a,#e(1,-1,#c)";
}

then when i parse the string, it messes up if you have a " within your message

Xx

i dont know why i posted that Xx


i would like to see tokenize3

[MysticHasteP2P_2]

Anything within " & " Is a token so:

Bob "went to the" market
tokens being
1-bob
2-went to the
3-market

[G_yoshi]

why not use tokenize2?

tokenize2 delims,str; - divides str into tokens, using 'delims' as additional delimiters

example:
tokenize2 $,Bob$went to the$market;

that will return a token for each one
#t(0) = Bob
#t(1) = went to the
#t(2) = market

Why add tokenize3 that does the samething as tokenize2? Delims, as indicated from the commands.rtf is the delimiter you want the command to use to seperate each word in the string. If you make delims blank then yes, each space in str will be the cutoff point each time it occurs.

[mikepg]

Originally posted by G_yoshi
why not use tokenize2?

tokenize2 delims,str; - divides str into tokens, using 'delims' as additional delimiters

example:
tokenize2 $,Bob$went to the$market;

that will return a token for each one
#t(0) = Bob
#t(1) = went to the
#t(2) = market

Why add tokenize3 that does the samething as tokenize2? Delims, as indicated from the commands.rtf is the delimiter you want the command to use to seperate each word in the string. If you make delims blank then yes, each space in str will be the cutoff point each time it occurs.

Nope, tokenize2 always has spaces and commas as default delims. so tokenize2 $,Bob$went to the$market;
would come out
0: Bob
1: went
2: to
3: the
4: market

Tokenize 3 might be useful...but you can easily get around the spaces with quotes....i use them all the time, for things like the on screen radio, it used wraptext, everything was just quoted though. It should work fine.

[G_yoshi]

Originally posted by mikepg


Nope, tokenize2 always has spaces and commas as default delims. so tokenize2 $,Bob$went to the$market;
would come out
0: Bob
1: went
2: to
3: the
4: market

Tokenize 3 might be useful...but you can easily get around the spaces with quotes....i use them all the time, for things like the on screen radio, it used wraptext, everything was just quoted though. It should work fine.

you specify 'delims' as the delimiter, did you not read the command description?

divides str into tokens, using 'delims' as additional `delimiters

[Falcor]

Originally posted by G_yoshi


you specify 'delims' as the delimiter, did you not read the command description?

divides str into tokens, using 'delims' as additional `delimiters

Case rested.

[G_yoshi]

Originally posted by Falcor


Case rested.

Eh?

I just fail to see the reason to duplicate an existing command. If it fails to work properly, then post it in the bugs forum Maybe Stefan will go back and make sure it works for the next release

[Warcaptain]

Originally posted by G_yoshi


Eh?

I just fail to see the reason to duplicate an existing command. If it fails to work properly, then post it in the bugs forum Maybe Stefan will go back and make sure it works for the next release

tokenize2 +,bob+went to+have+sex;
#t(0)=bob
#t(1)=went
#t(2)=to
#t(3)=have
#t(4)=sex

MY way
tokenize3 +,bob+went to+have+sex;
#t(0)=bob
#t(1)=went to
#t(2)=have
#t(3)=sex

tokenize3 wouldnt do the ADDITIONAL delimeters it would just simply use new delimeters and remove all old forms.

no spaces etc...

[Warcaptain]

Originally posted by Python523

Last I checked when you tokened something in quotes, it took everything inside the quotes as 1 token, I'll check again just to make sure later =/

no..
go on my server and type "/me test,test,test

only test will show up

and somewhere along the line, a line of only test will show up.

it uses #I()

which is same as tokenize2 without using command

[G_yoshi]

Originally posted by Warcaptain


tokenize2 +,bob+went to+have+sex;
#t(0)=bob
#t(1)=went
#t(2)=to
#t(3)=have
#t(4)=sex

MY way
tokenize3 +,bob+went to+have+sex;
#t(0)=bob
#t(1)=went to
#t(2)=have
#t(3)=sex

tokenize3 wouldnt do the ADDITIONAL delimeters it would just simply use new delimeters and remove all old forms.

no spaces etc...

Again, you've ignored what I said. If tokenize2 is not working right then report it! Don't be stupid by requesting a new command that is just a complete duplicate of an existing command.

[G_yoshi]

Originally posted by Warcaptain



no..
go on my server and type "/me test,test,test

only test will show up

and somewhere along the line, a line of only test will show up.

it uses #I()

which is same as tokenize2 without using command

#I() is for string arrays string arrays != tokenize2 :x

[Warcaptain]

Originally posted by G_yoshi


#I() is for string arrays string arrays != tokenize2 :x

but its the same as if you did:

tokenize2 ,,#s(stringname);

[which is what i did untill i found out about #I()]

[G_yoshi]

Originally posted by Warcaptain



but its the same as if you did:

tokenize2 ,,#s(stringname);

[which is what i did untill i found out about #I()]

Still, if tokenize2 is not working then why have you not reported it? Use your head

[Warcaptain]

Originally posted by G_yoshi


Still, if tokenize2 is not working then why have you not reported it? Use your head

tokenize2 is working....

its dividing things up into delims of spaces and also

NPC Code:

---

// NPC made by Warcaptain -Leader- (Abaddon)

if (created||timeout) {

  setstring test,bob,had,sex today;

  tokenize2 ,,#s(test);

  for (i=0;i<tokenscount;i++; ){

    showimg i,@Kristen ITC@c@Token #v(tokenscount-1-i): #t(tokenscount-1-i),playerx+1.5,playery-(i*1);

    changeimgvis i,2;

    showimg i+tokenscount,@Kristen ITC@c@Token #v(tokenscount-1-i): #t(tokenscount-1-i),playerx+1.5+.1,playery-(i*1)+.1;

    changeimgcolors i+tokenscount,0,0,1,0;

    changeimgvis i+tokenscount,1;

  }

  timeout=0.05;

}

---

try that out mister muster

[G_yoshi]

Originally posted by Warcaptain



tokenize2 is working....

its dividing things up into delims of spaces and also

NPC Code:

---

// NPC made by Warcaptain -Leader- (Abaddon)

if (created||timeout) {

  setstring test,bob,had,sex today;

  tokenize2 ,,#s(test);

  for (i=0;i<tokenscount;i++{

    showimg i,@Kristen ITC@c@Token #v(tokenscount-1-i): #t(tokenscount-1-i),playerx+1.5,playery-(i*1);

    changeimgvis i,2;

    showimg i+tokenscount,@Kristen ITC@c@Token #v(tokenscount-1-i): #t(tokenscount-1-i),playerx+1.5+.1,playery-(i*1)+.1;

    changeimgcolors i+tokenscount,0,0,1,0;

    changeimgvis i+tokenscount,1;

  }

  timeout=0.05;

}

---

try that out mister muster

*sigh* -_-

Get off your lazy ass and report it if its not working like it is supposed to!

[G_yoshi]

Now please, stop your complaining. I've done for you what you should've done when you first encountered the problem Next time, please think through all your options first. The last thing needed is a duplicate command

[Warcaptain]

obviously you arent listening....

it IS working fine.

but i want a command that doesnt use spaces at all when tokenizing.

so you dont do "s around the token to use spaces

[emortylone]

Argit. There isn't MUCH point to this... x.X If you feel it is necessary to do this, just use arrays!
setstring client.text,;
addstring client.text,bob;
addstring client.text,went to;
addstring client.text,the market;
That way you get the following:
client.text[0]=bob
client.text[1]=went to
client.text[2]=the market
Simple as that... there isn't a MAJOR purpose for your explanation of tokenize3 either... there are ways around it.
---Shifter

[Falcor]

uhm, Variables can't hold characters. Atleast not in Gscript. I think you have a warped understanding of Graal string arrays.

[Warcaptain]

Originally posted by Falcor
uhm, Variables can't hold characters. Atleast not in Gscript. I think you have a warped understanding of Graal string arrays.

client.text[0]=int; for numerical arrays is the equivilant of
insertstring client.text,0,alphabetical;
i know i use it alot on enigma for msgsystem

[konidias]

LoL GYoshi, you look like a big ass.

tokenize2 does what it is supposed to. He wants a tokenize3 that does not use " " and "," as delimiters. tokenize2 does, but it is supposed to.

But like explained earlier, if you did

string as: Bob "went to the" market

and used tokenize2, it would come out as

token1: Bob
token2: went to the
token3: market

So that is what you wanted, wasn't it?

[G_yoshi]

Originally posted by konidias
LoL GYoshi, you look like a big ass.

tokenize2 does what it is supposed to. He wants a tokenize3 that does not use " " and "," as delimiters. tokenize2 does, but it is supposed to.

But like explained earlier, if you did

string as: Bob "went to the" market

and used tokenize2, it would come out as

token1: Bob
token2: went to the
token3: market

So that is what you wanted, wasn't it?

Sorry...but this thread is a little stale, btw...I saw my error long ago