JSON Schema
Replies: 7 comments 12 replies
-
|
Let's try to answer these points as factually and precisely as possible:
This is an academic paper, an empirical study in the field of software engineering. Its scientific goals are to:
The last part or the paper is a new data structure description language based on JSON, which was initially a separate paper but got unfortunately merged there at the request of the program chair. The goal of the paper is not to promote JSON Schema, it is to study it, and to provide evidence-based feedback about the language design.
About the rational and controversies/disagreements so far, on each parts:
You do not need to take all the suggested spec changes to achieve some of the benefit. There is a continuum of spec changes which can forbid some things in the syntax, and others in the semantics, or keep the errors because you think it does not need to be addressed as it would harm some use cases thus are not worth the benefit.
Dunno!
It seems that currently rules depend on who you are: advocating breaking changes is okay for the spec team, but not so for outsiders with a differing agenda (in our case, an academic study: we are not serving the JSON Schema Community, we are not paid by a company which has a vested interested in promoting this technology because it is selling services based on that, we are independent, which is a requirement for academics). |
Beta Was this translation helpful? Give feedback.
-
|
I don't have a problem with posting content on the blog that is critical of JSON Schema. We have plenty to be critical of. But, such content should be the result of discussion including experts as well as the general community to make sure the criticisms are well founded and all perspectives are taken into account. Consider it our version of peer review. That certainly didn't happen with this paper. After reading the paper, I found there to be several inaccuracies, misunderstandings, and misrepresentations of JSON Schema as well as good deal of opinion and conjecture. Before I think this paper can be promoted on our blog, I think the problems should be corrected and the opinions should be subject to discussion and ideally all sides being represented in the blog post. I'd be happy discuss the parts I think are problematic, but doing so would be a considerable investment of my time and I'd like to know that you're willing to update the paper if I make that investment.
JSON Schema is a programming language. It's a declarative, domain specific language for the domain of validating an annotating JSON data. In every programming language there are ways to write nonsense programs including programs with unreachable code, programs that always exit in failure, or even programs that never exit at all. Sometimes errors are relatively easy to detect, other times they're complicated or impossible to detect, and sometimes it's ambiguous whether something is an error or intentional. JSON Schema is no different. Just like with other languages, our position is that this sort of error detection should be the responsibility of static analysis tools (linters). There have been a few attempts to make a linter, but nothing has been done that's terribly complete or widely adopted. Many of us consider it a huge gap in the JSON Schema ecosystem. It's something we very much want to tackle, but haven't had the capacity for yet. Although I don't find your recommendations very useful, your data is a very welcome contribution. It helps us identify what the most common errors are so future linting tools can look for those kinds of errors.
You could have asked. We'd have told you.
I think that your conclusion that the number of typos is related to the number keywords available to users is conjecture. Your data only shows what kind of typos happen. They say nothing about why those typos happen. You correctly call your conclusion "opinion", but I don't agree that this one is evidence-based. If you compare the number of typos across versions of JSON Schema and see a correlation of fewer typos in earlier versions that have fewer keywords, that would constitute evidence worthy of mention and further study. However, I see no indication that such an analysis was performed. However, even if it's true that more keywords means more typos, suggesting removing lesser used keywords isn't helpful. We don't introduce new keywords sitting behind closed doors that we think theoretically might be useful. We are very selective about what keywords we add to JSON Schema. We get requests for new keywords all the time and we reject almost all of them. Every keyword added to JSON Schema was added to address a clear and pervasive need from actual usage and user feedback. Removing a keyword means reintroducing a problem what we've already solved and making JSON Schema less expressive and harder to use. That's not really an option.
I presume you're referring to Greg's post about no longer allowing unknown keywords. That was a topic that we discussed and debated the details of for about a year over all. We discussed the issues internally as well as actively soliciting external feedback as far and as wide as we could including running polls. After all that, we came to a consensus and Greg volunteered to write a post for the community to summarize what all that work amounted to, how we came to our decision, and how the decision would effect users. You wrote a paper without soliciting feedback from the experts and have engaged in no community discussion. That's a very different situation. I don't expect you, as an "outsider", to have to go through the same level of diligence that we went through before Greg wrote that post, but I think asking for some level of discussion about a controversial topic is more than reasonable. After all sides are heard and understood, if you want to write a blog post summarizing the discussion and presenting all sides of the argument (assuming an agreement wasn't reached), we would love to have you write that post. This is no different from what we would expect of any "insider" who wants to write a controversial post. For the record, although some of us are paid to do this work, we represent the interests of the JSON Schema community, not the company that employs us. We're no more beholden to or influenced by the company that employs us than you are to the university that employs you. |
Beta Was this translation helpful? Give feedback.
-
The paper has been peer reviewed by researchers. Indeed, it has not been reviewed by the JSON Schema people, but this is not a requirement from our perspective, although it is fine. We read in detail most research papers about JSON Schema.
Please elaborate your claims, otherwise this is empty:
We tried to do a short blog post presenting the main results of the paper. The (interesting) discussion/debate you propose would be a longer one. Why not.
The paper has been accepted at a conference after being peer-reviewed. We are fine/would be happy to update the research report based on a detailed review by other people, if we find it appropriate. We may plan to submit an extended version to a Journal.
Hm... No, not really, IMHO. It is rather a less ambitious data description language.
Yes.
Mostly yes, but JSON Schema is not a programming language as such. Probably most data languages may allow some nonsense stuff at the syntax level, but much less so than a programming language because their power of expression/description is usually reduced compared to a full-fledged programming language, so there is less leeway for errors.
Sure. Our evidence shows unintentional errors in a majority of public schemas, without ambiguity in most cases.
Ok, this is your position. Too bad, no such linter is available for JSON Schema after 14 years and 10 drafts. There is a gradation of checks that can be performed about a DSL, from the syntax to the semantics to external tools: the current design puts very few constraints at the syntax/semantics level and all on a hypothetical linter. We indirectly prototyped one for our research. This research demonstrates that reasonable constraints on the syntax would detect most defects found by our tool in public schemas, a majority of which were proven defective. Our opinion is that the language design should be updated significantly in regard to these findings. We are fine if you want to keep your language as is.
Ok. Our opinion is that you could/should tackle the issue from the spec instead of from the tooling, and this opinion is backed by evidences that this approach works for its/most purposes: we tested the reduced language, it detects most defects, and it is practical. Now you may argue that these constraints are unbearable/unpractical/bad/whatever, but we are yet to be provided with concrete examples and use cases that can be investigated and discussed.
Ok... so what? The recommendations do work for avoiding defects, at the price of changing the spec.
Good! The paper basically describes the defects we investigated, and all our tooling is available online as public domain, feel free to look at, use and improve them.
Please do proceed, we are actually really interested on that point. Up to now we have not found any actual concrete example of use cases which would be made impractical with the restrictions we have suggested. If a few examples are found, then the cost/benefit of covering them with more or less convenience could be debated, but for now we just had believe me, which is not factual enough.
Hm. Maybe. It may just be that a language which silently ignores typos makes it likely that typos are not found, but then we are back to the syntax and our suggestion that it should be much more rigid. ISTM that there is a cluster of mistaken keywords around min/max related keywords, though, that is why we proposed this explanation. As a professor, I have experienced that the more student have to memorize things the harder it is for them. As an engineer who has practiced dozen (dozens?) of programming language in the last 40 years, memory is an issue. Ok, maybe age as well:-) Research shows that the human has a limited memory capacity (eg working memory, Miller 1956...). On the other hand, it is good to be able to give a word to a concept so that it can be named and discussed. So it is complicated. My opinion is that 60 keywords for describing JS data structures looks like a lot anyway.
Yes, and no. We think that these typos happen because THEY ARE NOT DETECTED by tools (validators, ...). In a normal development process, when you have a typo in a programming language, the program does not even compile or cannot run, so usually it would not be committed, and if it is committed it would be fixed at the first occasion and the dev would have an earful from the lead so they would strive to do clean commits. This auto-correction behavior does not happen with JSON Schema because typos are fine wrt to the spec. This is basically what we are suggesting to change.
Please pinpoint where our reasoning is wrong, so we can improve/correct the arguments and/or conclusions.
Interesting point. ISTM that we do not have enough data for that type of analysis.
We suggested to remove keywords which are seldom used and present implementation challenges which do not seem justified for the purpose of describing data structures. Now if typos are not ignored and types are checked in the syntax, the issue about typo errors would be fixed, so the argument about reducing these is much weaker. Anyway, this is not the main point of our recommendations.
We partially disagree: it is not obvious that more expressive is better, it should be a case by case discussion: this feature allows to write that, which is simpler than that (maybe, define simpler), but it adds these constraints and complexity in the implementation, and so on. Probably you did that with the 60 keywords, but the resulting spec is quite complex and people usually write schemas with defects (fact), and 1/3 of keywords are nearly never used in the public schemas we found (fact). Note that we also suggested to add a keyword to combine object properties, which would he a significant help in handing inheritance, a use case which does not seem well served by the current proposal.
Yes, that is one example, there may be other posts as well.
Good.
This is debatable: The research process involves feedback from academic colleagues and anonymous reviewers which are experts in their field of research.
Well, it is what we are doing now by proposing a blog post and discussing the paper on the side.
Please do not presume about the level of diligence in the academic community!
Sure. We are doing that. We have tried to present and list the controversies in our first post above in this discussion.
This is noted. We'll see.
In research, the fact that you are paid by someone who has an interest in your results requires an explicit declaration because it taints the results. This cannot be helped. |
Beta Was this translation helpful? Give feedback.
-
|
Ok, I'll start addressing some of the issues I have with the paper in separate threads, but there's something I need to ask of you. This conversation has so far not been productive. There's aggressive and defensive language coming from both sides and that needs to be reined in if anything productive is going to come out of this discussion. I promise to engage with you from a place of respect for you, your experience, and the work you've put into your research. I ask that you do the same for me. The goal should be learning from each other and understanding each other's position, not proving each other wrong. If we get to the point where I feel like we can no longer communicate from a place of mutual respect, I'll disengage from the conversation. I want to respond to a few things briefly, but I'll start an new thread for each topic I want to bring up.
There's some difference in our use of the term "programming language" here that I want to clear up. My understanding is that domain specific languages and general purpose languages are both categories "programming languages". However, you seem to have responded in a way that indicates that you thought I meant "general purpose" programming language. I'll address your classification of JSON Schema as data description language later, but I wanted to briefly address that miscommunication.
This is the kind of thing that could be taken as a passive aggressive attack that I would like to avoid in our up coming discussions. However, I'd like to provide you with some context about how why less has gotten accomplished in 14 years than you might expect. Until last year, JSON Schema has had no funding whatsoever. Everyone working on JSON Schema does so in their free time and takes on only what they feel they have the capacity to take on. Building a linter is a daunting task and it's unfortunate that no one as been willing and able to volunteer to take it on, but that's not something we can do much about without funding. Last year we were fortunate enough for a company to step up and support our work by employing several of us to work on JSON Schema full time, so you can expect more out of us in the next few years. Producing a linter is one of our highest priorities behind getting the next version of the spec out. A little more history as context. JSON Schema has had a rocky history. There have been multiple different groups working on the spec, sometimes without anyone overlapping from one group to the other. The project even went dormant for years before someone took the initiative to start it up again. I think it's changed leadership five times in its history. We're hopeful that now that we have some financial support, we can be more consistent and produce more.
Yes and no. We're willing to make changes when it's important, but we err on the side of not changing things if we don't have to especially if the change isn't backward compatible. This is the the way the current group working on the spec see things, but that hasn't always been the case. However, it's always been the case that we don't like to change fundamental concepts that have been established long before any of us were around. You have to remember that JSON Schema is widely used in projects big and small in all kinds of industries. The problem with making unnecessary and/or incompatible changes is that it makes it hard for people to upgrade to the latest version. Because upgrading could break their application, people stick to the older version. That means that implementations need to maintain support for multiple versions that users might be stuck on. It's not a good situation for anyone. Your data shows this as significant fragmentation of versions in use. We think that people generally use the latest version when they start a new schema, but don't upgrade once the schema is created. We would find that a interesting subject of further research. We're currently in the process of moving to a different paradigm for developing the spec inspired by ECMAScript. This new approach requires any changes to be compatible. This means that upgrading to the latest version is safe and implementations only need to support the latest version. This works because the older versions are always compatible with the latest version. This approach is better for everyone, but does mean that we are limited in some of the kinds of changes we can make.
You're not wrong, but I think it's important to note that you don't need to memorize every keyword to use JSON Schema. As you say, the set of commonly used keywords is much smaller. You only really need to learn a few keywords to get started and be productive. You can look up and learn the rest as you need more sophisticated features. If you don't have need for a feature, you don't have to memorize it. Most programming languages (general-purpose or domain-specific) have the same kind of learning process.
I didn't mean to imply that your reviewers weren't experts. I have no doubt that they're a experts in their field. However, they're not the experts in JSON Schema. We are.
That's not at all what I meant! I'm sorry I didn't communicate that clearly enough. I just meant that I don't expect you to have to put up with months of discussion just to contribute your thoughts. If you invited me to do a guest lecture in your class, you wouldn't hold me to same level of diligence required in the academic community. You wouldn't expect me to have my lecture peer reviewed and diligently cited to a degree that would be expected if I was submitting a research paper. I'm just there to share my knowledge and experience. That's what I meant. I don't expect you to have to go through the level of diligence we expect for introducing a spec change if you're just here to share you research and recommendations. |
Beta Was this translation helpful? Give feedback.
-
I started working on a linter almost 3 years ago, including user-definable rules. Information about it can be found here: https://github.com/ssilverman/snowy-json#the-linter Note that I stopped playing with this a little over 2.5 years ago (stopping at the 2019-09 draft) due to time constraints. (It was just a “fun” project for me at the time, useful for creating a schema for a standards doc for the entertainment industry; there were no validators that worked well for me that existed at the time, so I wrote my own.) |
Beta Was this translation helpful? Give feedback.
-
Interesting, thanks for the pointer! It suggests some issues that we did not look for (eg we did not bother to check min/max consistency) and we did more type-oriented checking. It is really a linter, i.e. some reported issues are more about style than defects as such, eg |
Beta Was this translation helpful? Give feedback.
-
|
Hi this Ben, Community Manager serving JSON Schema Community. I am not an expert in JSON Schema so I am not going to join the Technical discussion going on in this issue, instead I'd like to start a parallel discussion regarding the benefits of this blog post for the JSON Schema Community: The first time I read the message it appeared to me that your goal was just to present a brief highlight of your study, your findings, and your conclusions and then leave the forum, like you really don't care about how the broader community will perceive those findings and if someone will be able to understand it to take some or all of the proposals forward to evolve the spec and/or the tooling. I am sure that that wasn't your intention but it is what I personally perceived. I see this as an opportunity to improve JSON Schema, and this is why I'dl like to collaborate with you on some elements of the blog:
I hope this makes sense. 🙏 🙏🏾 |
Beta Was this translation helpful? Give feedback.
-
|
Hello Ben. This makes perfect sense. The intention of the blog was to be an as short as possible teaser for the paper, so that people could decide whether they are interested in reading it or not, so the content is mostly in the paper, not in the post. We can understand that it can be perceived as rough and lacking context. We expanded it along the line you suggest (add context, caveats, expand defects slightly). The last part is a little unclear for now. Feel free to provide feedback about the updated version. We are always interested in how our work is perceived, whether people agree or disagree (well, especially if they disagree 😄), and are fine with discussing our results with anyone. If the paper triggers a lively discussion, from our point of view this is good. |
Beta Was this translation helpful? Give feedback.
-
Ok. You may have gathered that I'm not a soft spoken diplomat, so I must work on that. My colleague is much better at that, so with some care and active proof reading the situation should improve.
Fine with me.
The computer science usual acceptance is that a programming language is a language which is somehow Turing complete, whether it is GP or DS is irrelevant. A programming language usually includes a part about describing/declaring data structures, and a part about control flow. JSON Schema only focus on the data structure side. There is no control flow as such (you may argue about if/then/else, but this is rather a mean to express predicates about the data structure). A DSL may or may not be a programming language depending on what it does, there is no inclusion one way or the other.
Thanks for these contextual information.
As we have written kind-of-a linter (freely open source) for our paper, we can provide an opinion, and this objective seems rather elusive:
So even if a linter is available, it may not be widely used. A benefit is that the spec does not need to be changed explicitly, which we understand is a primary concern. The syntax-level language restriction strategy/ies that we have put forward through our proposal would ensure that conforming tools would detect many defects, and the implementation is easy to do and check (with the standard test suite), compared to constraint in the semantics (which means that all conforming tools would have to implement some kind of analysis) or through external tools (linter option above, they must exist and you have to use them). There has been a few counter arguments. Against our data and analyses:
About the proposed changes:
About the general logic: we claim that the changes we suggest fix the identified issues.
Thanks for the history. From the output, it was pretty clear that the road had been bumpy, so your explanations says why.
Ok, preserving backward compatibility is a usual and useful objective for such a project. This is somehow a change of direction as, up to now, the design seems mainly to have focused on allowing upward compatibility (eg accepting unknown keywords), and most past versions have often broken backward compatibility, including the next. As academics, we are not committed to keep fundamental concepts established a long time ago, especially if these concepts lead to error-prone schemas. We looked at how to try to improve the situation at the spec level. It is very unclear whether it is possible to preserve both worlds, but at least we have checked that a significant part of existing schemas already conform to our proposed changes.
If you introduce significant breaking changes, ISTM that the pain will be the same whether you go all the way or half way.
We understand that you mean to focus on backward compatibility, i.e. old schemas are accepted by newer versions, and that will be true after the next version, so this version would be the last to include breaking changes. This suggests to do all appropriate changes now. We are looking forward to a benevolent, constructive and productive discussion. |
Beta Was this translation helpful? Give feedback.
-
More specifically, our (my) complaint is that the reported defects are not JSON Schema defects, but are often defects people experience when hand-authoring JSON. It gets worse if you're writing schemas with YAML. Your example (from the blog post proposal) {
"type": "array",
"items": {
"type": "string",
"uniqueItems": true
}
}is an issue of a misplaced If I were to express this in YAML, it's a difference in indentation, which is arguably even harder to discern visually: type: array
items:
type: string
uniqueItems: trueIf this were a subschema nested deep inside another, or if there are a lot more keywords in the subschema, this could be virtually impossible to pick up. Therefore, normal software development testing practices are highly encouraged. We expect that schema authors are verifying their schemas using both valid and invalid data to ensure that the schema allows only what it should. (This kind of testing is part of why I developed my data generation library. I even had an issue opened about it recently.) The conclusion that JSON Schema is at fault is presumptuous on the idea that people always edit JSON correctly. That said, we have received suggestions from other sources to only allow schemas that make sense semantically. The implementation |
Beta Was this translation helpful? Give feedback.
-
Yes.
The above argument is unclear to us. This example (1) conforms to the spec, (2) is an obvious error, indeed probably triggered by hand editing the schema, maybe with a benevolent JSON editor which suggested a wrong fix. It has to do with JSON Schema because this kind of schema is allowed by the spec, so there is a clear link from the spec to the defect, obviously through the hand of some user. Our proposal is to rule out at the syntax level such a schema, because then all conforming tool would detect the issue and report it, thus the user would be likely to fix the problem. You can argue that you want to keep this feature for some reason to be discussed, but the link to the spec seems obvious: the spec gives total freedom, which means that it is very easy so write nonsense schemas without ever realizing it, hence a majority of (public) schemas contain defects, which is one of the facts we have demonstrated in our paper. From a philosophical perspective, we think that languages can help or hinder users which are using them, so we are looking for ways to help the imperfect user to write perfect schemas.
We do not fully follow the therefore. We agree that testing should be encouraged! However, extensive testing seems impractical and costly for any non trivial schema, especially as you must test every potential type error to check that no type constraint would have been overlooked anywhere, so you should test improbable values everywhere. That could double the number of tests to write. This testing approach moves the potential fix from the spec (once, shared, inforced) to every project (many, duplicated, voluntary).
We do not understand your argument, so we are trying to interpret here. All broken or not broken schemas we have found are valid JSON, which suggest that people do edit JSON correctly: probably the editor helps, and if not any tool which loads the schema would complain if the schema is not even a correct JSON file, so any JSON error is very likely never to be committed. What people do not edit correctly are schemas, because a somehow random JSON is often a valid schema, which bring us back to the spec.
Indeed, we are not committed to the project philosophy per se: we have just analysed the output and suggested changes to improve it. We understand that your point is that the spec philosophy is not a possible point of discussion (for the spec) nor a subject (for a blog entry).
This argument lacks factual evidence. We are yet to be provided any concrete samples and illustrations that could be discussed precisely on their merit, in particular what they would look like with a restricted JSON Schema, and whether it would be somehow better/worse/equivalent on various objective or subjective criteria. SummaryWe understand that you seem to consider that:
We would like to see these use cases so that they could be discussed wrt our proposals. |
Beta Was this translation helpful? Give feedback.
-
|
The first thing I want to cover is foundational concepts. You have classified JSON Schema as a data description language. While we are aware that many people try to use it in that capacity, that's not what it's designed for. When it is used for that purpose, people do encounter many of the difficulties the paper highlights. JSON Schema is actually designed to be a "data validation language". More specifically, it's a "JSON validation language" because it aims to validate JSON, not necessarily any data. I know this distinction may seem trivial, but when you view things from that perspective, I think it should help explain why some things work the way they do. (JSON Schema is also designed to be a "JSON annotation language", but that's not relevant to the paper, so we can ignore that aspect for the purposes of this discussion.) As a validation language, each keyword represents a specific assertion about the data. That means that a JSON Schema is collection of assertions to be applied to a JSON instance. A JSON instance is valid if it passes every assertion in the schema. An empty schema is making no assertions. In general, keywords are designed to assert one thing and be combined as necessary to make more complex assertions. For example, the This is why schemas are "loose" by default. Being "strict" implies an assertion that properties that aren't mentioned in the schema aren't allowed. We use the terms "open" and "closed" generally in the same way you use "loose" and "strict". It's "open" because it's open for extension using composition ( The above are the foundational concepts of JSON Schema. If you want to study problems inherent in these foundational concepts, that's fine, but it's not constructive to ask us to change those concepts. If we did it wouldn't be JSON Schema anymore. It would be something else. This foundation is going to lead to some good properties and some bad. It will work very well in some applications and be problematic in others. (I want to be clear I'm not saying all of you suggestions violate these fundamental concepts, just that any that are are not something we would consider changing.) Here's an example to illustrate where these properties work to our advantage. {
"type": "object",
"properties": {
"foo": { "type": "string" },
"bar": {}
},
"additionalProperties": false,
"if": {
"properties": {
"foo": { "const": "bar" }
},
"required": ["foo"]
},
"then": { "required": ["bar"] }
}Notice that the Notice that the Notice that the Notice the
I know it seems odd for additional properties to be ignored, but I wanted to point out that an important reason for that to be allowed is to support polymorphism. Let's say I have a Schema that represents a vehicle and another that extends that schema to represent a car.
{
"type": "object",
"properties": {
"isHumanPowered": { "type": "boolean" }
...
},
"required": ["isHumanPowered", ...]
}
{
"$ref": "./vehicle",
"properties": {
"make": { "type": "string" },
"model": { "type": "string" }
},
"required": ["make", "model"]
}(Note that the Let's say my application has an
{
"isHumanPowered": false,
"make": "Toyota",
"model": "Tacoma"
}The desired behavior is that the "make" and "model" properties of this JSON instance are ignored when validating against the vehicle schema. Any kind of vehicle should pass. We could also have a bicycle schema and that should be allowed as well. This is an example of a good reason to use an open schema, but your argument is well taken that this isn't always what people want and it's easy to forget to use
The paper correctly identifies that the reason the meta-schema is loose is to allow for extensions. We agree 100% that this causes more problems than it needs to to support an extension feature. We have a plan to address this in the next release. We consider it important enough to make a breaking change to address this problem. People will have two ways to define custom keywords. One is to use the vocabulary system to define their keywords and the other is to use the I'll have more next week. I hope this was helpful. |
Beta Was this translation helpful? Give feedback.
-
Indeed, most public schemas we have collected are really simple data/type descriptions.
Hmm… There are a few features that fall under validation rather than typing, but not that many… regular expression matching (depending on what is checked though), multiple of, contains, min/max about values (which could be argued that this is pretty much typing). Formats or size constraints about arrays or objects are much closer to types. A key feature of data validation would be to write assertions, which with JSON Schema mostly means using Typical features of data (value) validation are not available, such as:
This would require boolean and other arithmetic expressions (not available directly, although the boolean expressions can be implemented to some extent with conditions and combinators) and value references (not available at all). So our view about JSON Schema is that it is:
This differs significantly from the declared validation language outlined above.
We do not understand this point.
We understand (and already understood) this internal logic, but it does not follow that doing so is a good idea, and doing otherwise is a bad idea.
Yep. From what we have seen, most schemas would be better off with tight defaults, because people often intend tight but forget something, hence create a defect. Tight defaults should not mean that loose cannot be expressed, it just mean that you have to say it explicitly, so there could be a few more lines. If you forget it, you would be likely to find out while testing, so you would probably fix it, so the drawback is minimal, so our point is that it would reduce the number of defect anyway and still be usable, hence we suggested that as a change.
We think that our suggestions are both reasonable and effective, so from our point of view this is constructive, although a constructive which is not necessarily compatible with your principles. In particular, the changes we suggest are compatible with the current syntax, as it is a subset, even if it does not adhere to your foundational concepts. We understand that adhering to these principles is a prerequisite for any changes. Based on these principles, most of our suggestions can probably be rejected directly, although you may cherry pick a few things. The fact that our proposals to not adhere to these principles does not invalidates our claims about defects in existing schemas, the link to the spec which facilitates/allows them, nor that our suggestions are a possible fix.
We disagree: we think that a compatible subset of JSON Schema, already used by many schemas, is still JSON Schema, somehow.
We think that we provided a measure of the bad in the paper.
Well, they mostly do violate these fundamental concepts.
This conditional example is interesting for discussion.
No, not really. Your are projecting how you think the verification should be implemented, which is the naive procedural approach. The validator could detect redundancy and remove useless evaluations. Some redundancy does not mean less efficient validation. The benefit could be on the software engineering side because the schema is easier to understand and less likely to contain errors. About the example, it is pretty hard to understand because of the It seems that this 13 properties schema can be rewritten within our subset as a 14 properties schema (assuming that additionalProperties is false by default, and that ... says any type, possibly with some predef or reference): {
"oneOf": [
{
"type": "object",
"properties": {
"foo": { "const": "bar" },
"bar": { ... }
},
"required": [ "foo", "bar" ]
},
{
"type": "object",
"properties": {
"foo": { "type": "string" },
"bla": { ... }
}
}
]
}From our point of view this version is better because:
Note that if you add two In the real world, probably
Sure. Users would probably survive if they had to declare type explicitly in the
This is a interesting aspect. Any actual use case example where the data must not be typed would be very interesting to investigate (i.e. not foo/bar, but some concrete example where an untyped element is really what is wanted). From a language design point of view, optimizing a design around a rare case in real life is not rational: the logical approach is to optimize the common case and to fit somehow the rare case. Here the rare case is the default (nothing to write!) and the common case is harder.
Yep, the internal logic is consistent, but it does not mean that the other approach cannot be made consistent as well.
Well, we think that requiring an explicit type would be less error prone because it would flag many errors for the user. Also, comparing awkwardness is highly subjective: We find the
Yes, to support polymorphism in a substractive system? If the additional properties default was inverted, it seems that it could still work by adding the keyword? See below.
Yes and no: If the schema is close by default, you could simply open it? So the help is marginal? You can achieve more or less the same thing with close by default schema. Our proposal suggests an additional merge operator to combine properties, which is a very common use case, so in our opinion would deserve a keyword. That could give: {
"$defs": {
"vehicule": {
"type": "object",
"properties": {
"isHumanPowered": { "type": "boolean" },
...
},
"required": [ "isHumanPowered", ... ],
"additionalProperties": true // say you want it open
},
"car": {
"mergeObjects": [
{ "$ref": "/$defs/vehicule" },
{
"type": "object",
"properties": {
"make": { "type": "string" },
"model": { "type": "string" }
},
"required": [ "make", "model" ],
"additionalProperties": true // or possibly false
}
]
}
},
...
}The semantics detail of such a merge is quite specific, but it does work. It must be thought as a preprocessed property, which would be expanded into a standard object when the model is loaded, with precise rules to merge each object-specific properties, and the result is interpreted after the expansion. Note that a merge combinator may make sense in a substractive (open by default) system as well.
Sure!
We already gave our thoughts about the linter approach: too much effort, will probably not be used as it should.
Indeed, being faithful to foundational principles is not a relevant motivation for academics, and I'm not sure it should be taken by an engineer either: keep it if it works well, ditch it if it doesn't?
Ok. Just for the record, we have some reservations about the vocabulary approach. SummaryWe understand that your point of view is that:
You have presented arguments for some of the features and principles, and we think that we have responded about each. Basically, both additive and substractive approaches can work, i.e. provide a language to declare schemas with a certain level of expressiveness. Based on our evidence, we still think that the current approach is intrinsically error-prone. The foundational principles prerequisite seems pretty much enough to close the discussion and reject most of our suggestions and the corresponding blog entry as incompatible, because they would change the philosophy, even if the syntax is kept. From an academic perspective, we do not see any strong argument to consider that these principles are good and should be preferred to alternatives: the only argument seems to be that they are already there and that you want to keep them. We think that it is quite hard to improve the language without amending some of these principles. Finally, we do not think that it invalidates our paper data, logic or conclusions. |
Beta Was this translation helpful? Give feedback.
-
|
I entered this discussion with the following words.
Unfortunately, it seems that you haven't been able to move away from taking a combative stance in this discussion, so I've decided not to continue. I started with foundational concepts so that we can set a baseline for discussion. I wanted to make clear what JSON Schema is designed for, show an example of it's intended use, and make clear what kinds of changes we can and can't make. More importantly, I wanted set the stage for an open and honest discussion by acknowledging some of the shortcomings of the way JSON Schema is designed while also showing some of it's benefits.
Rather than following my lead, you took every sentence as an attack on your paper. This thread wasn't about your paper. This thread was about setting a baseline for a productive discussion. I was planning to discuss my issues with the paper once we've established that we can continue from place of respect for each other's work. I think it's unfortunate that that didn't happen. I still think we could have learned from each other.
I wasn't making any such argument. On the contrary, I was very clear that there are both good and bad properties that come from this approach. This is an example of you seeing and creating conflict where there is none which is the main reason I've chosen not to continue this discussion.
There's no way we can have a productive conversation if you refuse to accept that the thing that we maintain isn't what we say it is. Just because it's possible to view a subset of our keywords from a data definition perspective doesn't mean that's actually what they are or what they are designed for or even used for. Your data can't support the claim that 99.5% of schemas are used for data definition. A schema written from the perspective of validation can look identical to a schema written from the perspective of data definition. A large part of your dataset comes from schemastore.org. The primary use of schemastore.org is to provide IDEs and editors with schemas to validate configuration files. Although we can't know the minds of the authors, I think it's reasonable to assume that most of those schemas are written for the purpose of validation.
This type of disrespectful comment is another reason I've chosen not to continue this discussion. If you change the fundamental nature of a thing, it isn't the same thing, it's something different. You could define a language based on a subset of JSON Schema keywords. It could be visually indistinguishable from JSON Schema, but if it's workings are fundamentally different, it wouldn't be JSON Schema. You're not the first to propose a strict-by-default, data-definition, JSON Schema-like language. Several exist. Maybe the approach JSON Schema takes will turn out to be a dead end and something else will turn out to be better, but if it that happens, changing JSON Schema to be something else wouldn't make sense. Something new would have to replace it.
Your schema isn't an alternative for the example I gave. If you put it into a validator you'll see that the intended conditionally required property assertion doesn't work. There is a way to express the conditionally required property assertion without
A |
Beta Was this translation helpful? Give feedback.
-
There wasn't any disrespect intended anywhere in our answer, but debate, some disagreement, and distinct perspectives. I'm sorry that you feel that you cannot discuss with us. Care and time was spent to smoothen the answer, although provably not enough.
Not really, but it is clear that it is how you perceived our answer. The quoted sentence is slightly out of the two previous paragraph context. We did understand and acknowledge the principles you presented and offered our own perspective on each point, because we thought this is normal debating. Then we reflected that the changes we propose mostly do not conform to these principles, thus can be rejected directly with this filter without further discussion. As it is potentially conclusive, we thought about what changes compatible with the principles could help improve the language (well, this is very unclear) and about the impact on our paper, this later point matters to us because invalid papers should be retracted (in theory), so this question is legitimate.
I do not think that we claimed that JSON Schema is not designed or cannot be used as a validation language. However, we also gave our perspective about validation features and how much simple typing and advanced validating there is in the language and the corpus we studied. The good thing about perspectives is that there can be different ones about the same facts. Note that typing is a kind of data validation. Computer science sees a lot of things as typing because of the large body of knowledge built over the past 120 years, so it is very hard to root out that way of thinking for academics.
Sure. It seems that there is a misunderstanding: We have no idea whether schema designers thought they were doing validation or mere typing, and this is not what we are trying to achieve. We looked at the features they actually used in their schemas, at which values are accepted by these, and concluded what they actually do with JSON Schema is mostly to describe types, which is one (simple) kind of validation, although not the full fledged one.
The Store source is 653/57,803, slightly over 1%, thus a small part.
Sure, very probably. This is not contradictory to our point that these schemas are very often reducible to typing.
Indeed, this is very interesting! If-removal is touchy (hence the seems), and here it did not work because the first schema is a subset of the second. It could work this way if there was a discriminator, but here it needs an exclusion.
The exclusion can be achieved with {
"oneOf": [
{
"type": "object",
"properties": {
"foo": { "const": "bla" },
"bla": true
},
"required": [ "foo", "bla" ],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"foo": { "oneOf": [ { "type": "string" }, { "const": "bla" } ] },
"bla": true
},
"additionalProperties": false
}
]
}It is 12 keywords and 4 properties long, adding 2 keywords to the previous failed version. It seems equivalent to yours. |
Beta Was this translation helpful? Give feedback.
-
|
@benjagm: It seems that the discussion about the paper contents has stalled: Too bad, shame on us. We have updated the blog entry to highlight the disagreements as we understood them, add context and caveats, so as to possibly make it acceptable… Nevertheless, the blog entry is not about supporting JSON Schema as it is, thus is somehow controversial. Please consider deciding to: (1) accept the blog entry with its controversies (2) ask for more changes… (3) ditch the entry because you do not want dissenting opinions shown in the community web site. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @zx80. As you know, we had a lengthy discussion showing everyone's efforts to learn from each other, so first of all we'd like to thank everyone for the effort in engaging this conversation. After long consideration, in the last OCWM, we have decided not to publish the blog and encourage you to use other channels of the JSON Schema Community, like GitHub Discussions to continue the exchange of ideas, or the The JSON Schema's Blog main goal is to promote the JSON Schema adoption, and this is why the content needs to fulfill specific criteria as per the blog guidelines, and as said before, this was not the case. The JSON Schema Community is proud to be a diverse and safe space, and we encourage everyone to respectfully share their opinions, having in mind the JSON Schema Code of Conduct and here we celebrate diversity. We are just providing the appropriate channel for this type of content/discussion for this case. We are mindful that this can be frustrating at this point, but trust us, this was a difficult decision, and we took a lot of care in every step of the process. We hope you stay in the Community so we can continue learning from each other. |
Beta Was this translation helpful? Give feedback.
-
|
Hello @benjagm,
Ok!
Sure. It seems that these guidelines have been devised after the post was submitted, thus are applied retroactively.
Ok. The fact that only some people are allowed opinions in this community is unusual from an academic perspective. Anyway, we take good notice that the blog is exclusively about promoting JSON Schema by insiders, thus is not the right place to propose changes and provide opinions for others. Thanks for all the discussions which have greatly improved our understanding of how the JSON Schema community works. |
Beta Was this translation helpful? Give feedback.
-
|
I am sorry that my message has been perceived that way.
We already invited you to share your work in other channels of the JSON Schema Community, therefore is not fair to make that asseveration. We are proud to collaborate with other researchers in the past, but in the past the researching teams approached us during their research and before the publication of their papers, what created an ideal collaboration context. However, this wasn't the case.
We need contributors, but not at any price. My personal vision of Open Source is that the first step to become a contributor should be engage with the Community and that happens on Slack and Github, the channels we are encouraging you to use. Maybe in the academic domain the contribution path starts differently, and maybe this is causing this friction. I am going to use my last sentence to once again invite you to share your work and line of thinking with the JSON Schema Community starting a discussion in Github and posting in the |
Beta Was this translation helpful? Give feedback.
-
Hi everyone!
We recently received this blog proposal PR by researchers external to JSON Schema Community sharing defects found in public schemas. Even though we are excited to receive new contributions in any possible way, the content differs from the initial purpose of the blog, and we are unclear about the goals of the publication.
We are respectful with everyones work, and we recognize the effort behind the study backing the article, however we'd like to start a conversation between authors and the Community to make sure we publish content that supports the JSON Schema Community the best way.
How to start?
Who can participate?
We are expecting primarily the blog authors and JSON Schema maintainers, however everyone is invited and we'll love to have more opinions about this.
Beta Was this translation helpful? Give feedback.
All reactions