Parsing 101

notthecheatr · Master

I wrote a simple object called words_list which takes a string and splits it into words. Its main use would probably be parsing, since for example it'll take something like this:

1000101 · Hero

I hate to be a buzz-kill, but...strtok()

duke4e · Sr. Member

strtok()
what the hell is that?

Peter · Posted: Dec 15, 2007 3:21 Post subject:

strtok = String Tokenize, basically it breaks down a string into the component pieces i.e. words. It pretty much does what your trying to do with your code. That said, I think your code is great, it gives you the basis to make much more complex stuff that is (and I think this is important sometimes) completely independent of outside calls.

Back to lurkerdom for me now.

MichaelW · Hero

For general-purpose use your whitespace should probably include tab characters. Strtok is fast and easy to use, but a byte scanner is probably a better choice where you need the length of each token and/or where you need to detect errors such as unmatched quotes.

Finding Tokens in a String

notthecheatr · Master

Now I know why people complain.

Actually, it's mostly a learning thing. Like I said, the code's pretty much a hack, but it might be useful and that's why I wrote it. I don't really care if there's something else that already does that, as using that other tool won't really teach me anything. Sure, it's good to use other tools, but there's nothing wrong with writing your own thing, is there?

P.S. This is an object. I could write an object based on strtok() but in any case it doesn't matter... my point is that using an object makes it a much more natural interface.

Peter · Posted: Dec 16, 2007 2:43 Post subject:

yetifoot · Master