Add more lex tests

[JakeRoggenbuck]

No description provided.

[adamhutchings]

Would it be possible to get some representation for failure paths, e.g. invalid tokens?

[JakeRoggenbuck]

What do you mean?

[adamhutchings]

I mean as in testing that it fails gracefully and doesn't segfault or something if we ask ttype_string("/* This is not a token. */") or similar.

[JakeRoggenbuck]

Yea, testing doesn't segfault, it just prints the assert that failed

[adamhutchings]

I don't mean that the testing code itself might be broken, I mean testing whether ttype_string might fail gracefully or just crash or something. I think this is good to test because our system should be robust.

[JakeRoggenbuck]

Ohh I see, yup we definitely should have that added. Same PR or later?

[adamhutchings]

Later is probably fine.

[NicoBliss]

do we know that everything not listed is an identifier?

[JakeRoggenbuck]

Great question. I think we might need to check for one character symbols that definitely aren't tokens or identifiers. I can only think of things like the @ symbol that isn't either, so we might want to have error handling for that later

[NicoBliss]

testing this function is great! I am worried that as a lookup table with no information about the ordering of the enum, it is a bit fragile though.

[JakeRoggenbuck]

That's true. Having this is a good canary in the coal mine for names not working. If the tests break, any code that relies on the function will also break. It's a problem with how naming enums works in C in general so we should be careful with the use of the enum + name function.

[adamhutchings]

Looks good, but merge conflicts :(

[JakeRoggenbuck]

Looks good, but merge conflicts :(

Oh noooo

[JakeRoggenbuck]

I will fix

[adamhutchings]

Assuming the tests pass (which see PR #40 for why I'm not testing this myself), this looks good!

[JakeRoggenbuck]

Yup, tests pass on this one. Tests however fail on main.

jccc (add-more-lex-testing) λ ./scripts/build.sh
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /home/jake/Repos/jccc/build
[  6%] Building C object CMakeFiles/jccc.dir/src/lexer/lex.c.o
[ 12%] Building C object CMakeFiles/jccc.dir/src/lexer/test_lexer.c.o
[ 18%] Building C object CMakeFiles/jccc.dir/src/driver/main.c.o
[ 25%] Linking C executable jccc
[ 50%] Built target jccc
[ 56%] Building C object CMakeFiles/test_jccc.dir/src/lexer/lex.c.o
[ 62%] Building C object CMakeFiles/test_jccc.dir/src/lexer/test_lexer.c.o
[ 68%] Building C object CMakeFiles/test_jccc.dir/src/testing/main.c.o
[ 75%] Linking C executable test_jccc
[100%] Built target test_jccc
jccc (add-more-lex-testing) λ ./build/test_jccc
Running tests from "test_lexer" ...
Running "test_ttype_name"
Running "test_ttype_from_string"
Running "test_ttype_many_chars"
Running "test_ttype_one_char"
Concluded tests from "test_lexer"
Running tests from "test_x86" ...
Running "test_init_int_literal"
Concluded tests from "test_x86"