The following Python code reads JSON data from two JSON files and checks if the data in these two different files are same or not. The two final contenders were rapidjson and orjson.Compare if Two JSON files have same Data Regardless of Order in Python
hyperjson only has packages for macOS, and in general seems pretty immature. 
It looks like it’s being maintained again, but it still seems pretty crash-prone, which suggests it’s still insecure.
At the time I originally wrote this, ujson had a number of bugs filed regarding crashes, and no release since 2016. I filtered out some of these based on the criteria above: Libraries I considered were orjson, rapidjson, ujson, and hyperjson. Maintained: I don’t want to rely on a library that isn’t being actively supported. Cross-platform: runs on Linux, macOS, Windows. Some JSON libraries support this, others do not. Custom encoding: Eliot supports customization of JSON encoding, so you can serialize additional kinds of Python objects. If the JSON encoder crashes on bad data, that is not good either for reliability or security. Security/crash resistance: log messages can contain data that comes from untrusted sources. Performance isn’t everything-there are other things you might care about. Step #3: Filter based on additional requirements I came up with the following sample message, based on some real logs: In my case I mostly care about encoding small messages, the particular structure of log messages generated by Eliot. Do you care about encoding, decoding, or both?. 
So you want to come up with some measure that matches your particular usage patterns: Quite often they’re measuring very large messages, and in my case at least I care about small messages. Those messages don’t necessarily correspond to your usage, however. If you look at the benchmark pages for various JSON libraries, they will talk about how they do on a variety of different messages.
The most speedup I could get is running 33% faster (if JSON encoding time went to zero), but that’s a big enough chunk of time that sooner or later it would make it to the top of the list. In my case, I learned this from a benchmark for my causal logging library Eliot, which suggested that JSON encoding took up something like 25% of the CPU time used generating messages. Just because you use JSON doesn’t mean it’s a relevant bottleneck.īefore you spend any time thinking about which JSON library, you need some evidence suggesting Python’s built-in JSON library really is a problem in your particular application. Step #1: Do you actually need a new JSON library?
Filter based on additional requirements. You can use this process to pick the library that best fits your particular needs: So to help you choose the fastest JSON library for your needs, I’d like to share the process I went through to choose a fast JSON library for Python. Speed isn’t everything-there are other things you may care about, like security and customization. A “fast JSON library” means different things to different people, because their usage patterns are different. The truth is there’s no one correct answer, no one fastest JSON library to rule them all: Python’s built-in library isn’t bad, but there are multiple faster JSON libraries available: how do you choose which one to use? The more you use JSON, the more likely you are to encounter JSON encoding or decoding as a bottleneck.
0 kommentar(er)
