JSON

Aug 21, 2020

After posting the Delores code, I read a comment from someone who lambasted me for not using compliant JSON for the .wimpy and other data files.

They are 100% right, I did not use compliant JSON for those or any of my JSON-like files.

JSON was created by Doug Crockford, whom I worked with at Lucasfilm. Doug is a super smart guy and JSON is one of the best formats I've ever seen. Fully compliant JSON is also a pain in the ass.

Given that this was my engine, I decided to change it. For the record, my JSON reader can read compliant JSON just fine. With the right flags, it can also write fully compliant JSON.

Here is how my JSON differs and the reason for those changes.

1) Keys in a dictionary don't have the be quoted if they are simple alphanumerics. a-zA-Z0-9_

{
    foo: 100
}

2) Values that are simple alphanumerics don't need to be quoted.

{
    name: Delores
}

3) If a file doesn't start with a { or [ then it is assumed to be a dictionary.

foo: 100

The main reason for this change was for the Prefs.json file. I expect these to be edited by consumers and { } is hard to understand if you're not a techie person. I was seeing people add new keys after the closing }

{
    foo: 100
}
bar: 1

99% of my JSON files are dictionaries, so it seem fine to drop this requirement.

4) Commas are optional and ignored.

{
    foo: 100
    bar: 200
    list: [ 1 2 3 "four" 5 6 ]
}

There is no syntactical reason for commas. If you need multiple items on one line, then you use them.

{
    foo: 100, bar: 200
}

This is also OK. Commas don't matter

foo:100,,,,,bar: 200

The only thing I don't like about my custom format (and the reason for the initial complaint) is they can't be loaded into any of the excellent JSON editors.

That is true.

I guess everyone needs to come around to my JSON format.


Nor Treblig Aug 21, 2020
The one thing with JSON which annoys me most is that you cannot have missing properties, e.g.:
{
    foo: 100,
    bar: 200,
}

Do you allow comments? Most parsers I work with do, which is great.

Ron Gilbert Aug 21, 2020
I don't allow comments because most of my files and read then written and there is no way to store the comments,

Kennedy Rose Aug 21, 2020
Nice! Do you ever use Yaml? It reminds me of that a little.

Little Cat Aug 21, 2020
I feel like this is a lot better than the standard format we all know and love, easier to write by hand too.  Sure wish everyone would implements this kind of formatting.

Per Aug 21, 2020
I love your file format but please don't call it JSON, because it's not. Call it GSON or something.

Jay Kay Aug 21, 2020
Check out TOML (https://toml.io). "Human-usable" is the term they use.

Nor Treblig Aug 22, 2020
GSON is already used, but maybe GGON (GrumpyGamer Object Notation)!

When I handcraft JSONs it's either to figure out what to generate or configuration files.
For the latter comments are useful for quickly including/excluding data, switch stuff like connection strings around etc. while having them clearly formatted (what's data vs. comments).

Ingo Rogerson Aug 22, 2020
Hello Ron , All set and done. I am reading Tim Powers's Stress of her regard. Thank you for everything.

Nik Aug 22, 2020
You're part way to re-inventing HOCON - https://github.com/lightbend/config/blob/master/HOCON.md

grumpycats Aug 22, 2020
fully compliant or not, it is a great fun to browse thru the scripts, graphics  and other files of the code. Thank you for uploading.

Ron Gilbert Aug 22, 2020
Point taken about not calling it JSON.  GGON isn't bad.

I use YMAL a lot, but it's not a format I would never expect consumers to be able to correctly edit.  To the average person space and tab are exactly the same thing.

The other formats are mostly fine, but they still suffer from the problem of there not being a ton of editors that understand them.

Rich Suanders Aug 22, 2020
I prefer Python dictionaries over JSON for a few reasons.  One, I like to be able to use ' or " for my strings.  As a C/C++ hacker, I hated the '' for strings at first, but the lack of shift-key for the ' is something you get used it, as well as being able to put ' in " and " in ' much more easily.  
I always want comments: even if I don't store them, they can help me remind me of things # Remember, delores needs this later ...

Richard Saunders Aug 22, 2020
(Oh yes, with Python dictionaies I can either use text OR a binary pickle format which make a huge difference in transmitting binary numpy arrays.  People also forget that it's nice for over-the-wire to be able to put binary data in a file, and JSON really can't don't that without escaping all the binary chars)

Tom Aug 24, 2020
Re: comments - I guess you can still write { comment: "Who said anything about comments?"} right?The read/write cycle would just need to keep unknown keys and always write them back  then.

Oscar Aug 26, 2020
Your non complient style of being a programmer inspires me. I almost fell that i could start enjoying my work again!

Christian Aug 30, 2020
So, GGON had the opposite design goal of XML, which was created to be tedious to read by humans and machines?

Billy Frank Aug 31, 2020
I use a lot of human-edited but Python-parsed JSON for my games, and doing something like this would be tempting if I can figure out how to get Blender to use a custom parser.

Matej Sep 06, 2020
Given the rules you describe, it shouldn't be that difficult to write a simple tool that takes your files and converts them to a compliant JSON. Converting it back might be a bit more tricky, but not impossible.

In any case, you pick the approaches that work best to your use case, which is usually ther right way.

xmuskrat Sep 06, 2020
Have you ever checked out YAML?

jujojow Sep 06, 2020
How long did it take you to make your custom json parser? Do you think it was worth the time you spent doing it? I'm always cautious about the time it would take to implement such, but then if it reduces friction and speeds development and testing time later maybe such things are worthwihle.

xmuskrat Sep 07, 2020
To be fair, both JSON and XML were designed to be easy to parse, so probably not as much time as you'd think.

Shaz Sep 08, 2020
This is *almost* like HJSON https://hjson.github.io/ which  has a lot of package support for all the languages. For the last 5% that is missing I'm sure they will take in a pull request https://github.com/hjson/hjson

Alien426 4d ago
I guess I'll have to be the first to voice my misgivings. I think JSON is easy enough to understand, yet provides enough structure to be useful for data exchange. Your treatment of strings being handled equal to numbers is probably not a problem in your environment where the key dictates the type of the value, but generally I prefer the explicit distinction.

I could never stand XML for data files. Its use in RSS and Atom feeds (which are slowly dying) was always weird given how much overhead there is. And that it continues its life (in compressed form) in office files like ODG and DOCX still seems like an odd choice.

@Jay Kay
TOML is essentially just a INI file, isn't it?

Add your comment:


Here are the rules for commenting.