Query all fields of a type in a simple expression [wildcard]

[AhmadAudeh]

Let us say this is the type :

type Human {
  id: String
  name: String
  homePlanet: String
}

Then to query all the fields the query is :

query HumanNameQuery {
   Human {
     id
     name
     homeplanet
    }
 }

Should be something like this exist ??

query HumanNameQuery {
   Human {
     *
   }
 }

[rmosolgo]

I think this was left out of GraphQL on purpose. Although it seems convenient at first, it may actually lead to some maintainability troubles down the road. Consider some possibilities:

• A field is removed.

  Let's say the database changes and Human.homeplanet doesn't exist anymore. Any query Human { * } would execute successfully, but the client may encounter an error because it depends on homeplanet, which is gone now.

  By requiring an explicit list of fields, you get notified right away that a query is invalid -- you don't have to trace the error down through the client or UI.

• A field is added.

  Imagine that Human.friends is added, which returns a list of other Human objects. Does Human { * } include those objects? If so, any client which requests Human { * } but doesn't need friends has caused a lot of waste!

  It's better to request exactly which fields you need so that the server doesn't do any extra work. This also allows backend developers to implement resource-intensive fields without the fear that they would be requested by accident.

For those two reasons, it's better to make the client request fields explicitly!

[dschafer]

Yep, this was indeed deliberately omitted for the reasons @rmosolgo mentioned!

In places where we've needed things like this (a good example might be a Date type where you pretty much always want to fetch year, month, and day), we could create a fragment DateFields on Date { year, month, day } and use that everywhere that we would have used * before.

[AhmadAudeh]

@rmosolgo : thanks for your explanation, it makes sens.
@dschafer : seems a usable solution, I will try that, thanks.

[amunna]

Interesting reasoning to remove the "*". It's nice to explicitly mention the search fields but it wouldn't hurt to have an "wild card" option just to explore the fields. To me this is just limiting the possibilities with search and doesn't make sense to me.

[acjay]

@amunna I agree that it's awkward sometimes to issue a bunch of queries to explore an API, rather than have an easy way to see a sample of all of the data. But this seems to me like an issue that should be solved in clients like GraphiQL. It could automatically expand a * to be all fields that can be called without arguments. Which, by the way, is another wrinkle. GraphQL doesn't currently assign any special meaning to fields with or without arguments, so it's a little arbitrary that introducing a wildcard in the language itself would create that difference.

[MorganConrad]

@rmosolgo Your point#2 (adding a field) is a red herring. If the client cares about performance, they can still be explicit in their fields. If it is just an exploratory client who doesn't care about performance, they could use "*" (were it supported). Exactly how SQL has done it for a long time.

Your point#1 has some validity, but, again, if the client really cared, they could be explicit.

[fsc7]

One more to disagree with the absence of the possibility of a wildcard like '*' to retrieve all fields. This feature is very useful if you are exploring an API.

[tomprogers]

I want a wildcard so that I can fetch a dictionary from the server. This is not exploratory, it's a real case that comes up when internationalizing a web application.

I don't know the names of the fields, so this is not solvable using fragments.. Converting the structure to an array of key:value on the server and then back into a dictionary on the client is painful when using apollo tooling, and also makes it much clearer that I'm just working around arbitrary shortcomings of GQL semantics.

query {
  user {
    firstname
    lastname
    preferredLanguage
  }
  
  languages {
    code
    label
  }
  
  strings {
    * // I want strings to be a dictionary of key:value pairs
  }
}

[juancampa]

@tomprogers for those cases you might want to model your strings field as a list of key-value pairs

[PSAustin]

We need to return some very deep, fully hydrated, object graphs. Without this feature that task is very laborious. This could even be a deal breaker for our current project where we were planning to use GraphQL.

[stubailo]

I think this is a duplicate of: #101

The ideal solution here IMO is to introduce a way to return objects, rather than make all objects able to be queried without specifying fields.

[jlouis]

GraphQL currently has the property that the client is explicit in everything it queries. This suggestion would mean we lose this property. Thus, it has to be done with a lot of care: clients using * will eventually open themselves up to getting lots of data they didn't ask for. Also, the server has no way of knowing which fields might be good candidates for deprecation, if the * query is used.

You can always define scalar JSON and just return a blob of JSON in place of a query. You cannot query inside the structure, but it can be arbitrary, deeply nested objects returned from it.

Currently, the best bet is to model it with a parameter to the query:

query Q($locale : LocaleCode) {
    user(login: "foo") {
        givenName(locale: $locale)
        familyName(locale: $locale)
        fullName(locale: $locale)
    }
}

Personally, I think what is needed is some kind of "implicit parameters", so you can define an implicit binding over a Selection Set. That means, for every subquery, you have access to the implicit parameter, if it has not been overridden. In turn, you can set a locale high up the query and then have everything under in scope pick up the implicit param naturally.

This is akin to digging out a Accept-Language header from the HTTP query and stuffing it into the context of the GraphQL execution, using it as a default value.

As I've said elsewhere: GraphQL is a programming language. You send small programs to the server for it to interpret, and that language could as well contain a let-bind construct of some kind.

[acjay]

@jlouis

As I've said elsewhere: GraphQL is a programming language. You send small programs to the server for it to interpret, and that language could as well contain a let-bind construct of some kind.

^ This.

I wrote about this in detail: http://artsy.github.io/blog/2018/05/08/is-graphql-the-future/

[tomprogers]

you might want to model your strings field as a list of key-value pairs

@juancampa No, I don't want to model my strings that way. I thought I made that clear in my original post.

[mjmahone]

It would probably be more productive to add a concrete proposal on a new issue rather than re-hashing a nearly three year old, closed thread. Especially as it sounds like there are ~3-4 different potential problems and solutions that have come up here:

• Having "*" somehow be used for "explore available fields": I don't know what that looks like as part of the language specification (as opposed to a client utility), so I'd be interested in seeing a concrete idea around that. That idea should be careful not to compromise forwards-compatibility (i.e. as the server adds or removes fields, the response shape shouldn't change: this would indicate it should be tooling built on top of GraphQL rather than core to GraphQL's specification, I think).
• Figuring out performance vs. developer ergonomics around a "wildcard" approach.
• Fetching a dictionary from a server. This may or may not be best solved with a wildcard: you can use things like a "Dictionary" scalar field to do this. Additionally, there may be room to clarify in the specification how to resolve this (for example using key-value objects in a sorted array, which is isomorphic to a dictionary), but again, that seems like a different request than this issue.
• Defining variable bindings somehow (maybe this would involve fragment variable definitions?)

Each of these all sound like interesting ideas that should be discussed! I'd be interested in digging deeper into each of these proposals, but it's difficult to do without having the discussion derailed when there are many different threads pulling apart on a closed, only-loosely-related issue.

[Danilo-Araujo-Silva]

This is somehow nonsense. When we are developing and we need to inspect a table on a database, for example, we don't write the 20 columns a table may have, we use "*"? ^^
Here should be the same.

[unleashit]

Plus one. Learning that a fields wildcard isn't possible, and that the thread to request has even been closed is a disappointment. I understand the reasoning for wanting to keep it explicit, but there are times, especially when one is testing or wanting to develop quickly, that it would be no less than a god send to be able to say "just gimme all the fields". The fragment doesn't help with this use case.

What about a possible solution of a flag on the server that enables support? The default could even be not to accept if you wanted to keep it strict.

[Jaikant]

I was searching for a wildcard option as well, sad it isn't yet available.

[brian-1901]

To be honest, you can make fragments work which leads to better organization of your code. 🤷‍♂️

[unleashit]

No need to lecture friend. Sometimes better organization isn't the goal. Sometimes speed is. Read my comment again. If you feel like over engineering should be enforced on everyone and for all use cases, you have the right to your opinion but we disagree.

[brian-1901]

Guess if your goal is speed, than my comment doesn't apply to you.

[rodrigopires]

Plus one for having the wildcard possibility implemented.

[FrankSD]

This is non-sense.

[winni4eva]

This issue was closed with an issue that linked back to this issue..nice.

[jdspugh]

NoSQL databases are based on graph data structures rather than tables. We should be able to query any graph node in a NoSQL database and receive its entirety, including children, to the depth we want with its query language (GraphQL). If you think in graph terms, you are simply returning a graph node and its children which is one of the most fundamental, basic and efficient operations on any graph data structure.

Anybody used jQuery? It's a graph query language also. The most common query operation $('#node-id') returns the object, its attributes and its children without specifying what the attributes and children are upfront. This should have an equivalent in GraphQL.

[armaandh]

I agree with the suggestion that at least while developing..we should have a way to be able to quickly wildcard all the fields. This is a nice productivity boosting during rapid development, and does not necessary imply it should be something used in production.

[boazsegev]

I'm fairly certain that exposing an "all fields" (*) operator should also be considered a security risk when exposing large data-sets.

Seriously, what is the amount of data provided by a User(id: X) {*} query on facebook? all posts, all likes, all links... how much data would that be? How easy would it be for a client to mistakingly send such a query?

I'm just starting to learn GraphQL and I wanted a * operator for my exploration... but I think the world is safer without it.

[gwardwell]

@SephReed

Learning, on-boarding, self-discovery; all worthless

Just so we’re clear, I never said learning, onboarding, and self-discovery are worthless. I said implementing features to help with discoverability belon in user land and other tooling. There are many options out there to make learning easy. New and better ones are appearing all the time. Take the Prisma Playground app for example. It’s far better than using command line, even with a wildcard, for learning about a schema. Why should a feature be added that is only for onboarding and very risky for production applications when there are so many tools to achieve the same thing? If the argument is that they are difficult to set up, then go work with your favorite one to make it easier. If the argument is that they’re not discoverable, then go work on spreading the word. But to say that a feature solely meant for onboarding needs to be added to the official specification just doesn’t make sense to me.

If anyone wants to go on record saying they're incapable of coding a recursion function with a depth limit, then we can all agree they can't do it. Still wouldn't change the fact that the code is absolutely achievable. The honest real blocker is that it's a feature for noobs.

Again, not at all saying new devs shouldn’t be supported. They should be. Every time a developer adopts GraphQL an angel gets their wings and the internet becomes a much better place! But adding a wildcard to the spec is not the way to do it. And I will go on the record and say this is not an easy ask. But not because of recursion. There are a lot of variables. If it were just a recursion loop it would admittedly be easy. @sungam3r wrote up a pretty good list of what it would take to make this request happen in this comment. That list isn’t reasonable when the only reason those changes, tooling, and rewriting have to be made is to make self-guided onboarding easier. Especially when, as I’ve said several times, there are far better tools for that out there.

One thing I just now noticed that has gone unmentioned this conversation are introspection queries. Those are already part of the spec. Does that achieve at least part of the goals you’re looking for with wildcards?

[mike-marcacci]

@SephReed - Mocking those who have donated tremendous effort to create an extremely robust and productive spec is inappropriate and unproductive. Invoking the "old boys club" power dynamic to describe this community is a wild misapplication of that term, and "anti-educational" and "elitist" are properly rediculous labels for an open source technology with an almost distracting level of entheusiastic proseletizing among its rapidly growing userbase.

If you truly feel these labels are justified, I'd like to invite you to join the upcoming working group meeting (and nobody needs an invitation, by the way) so you can voice your concerns.

I think you'll find that everyone involved cares deeply about the interests of the community and the future of the technology.

---

To summarize the points that are already well described in the long thread above:

1. This isn't a straightforward request, even if it sounds like one.
2. The presence of such a feature would likely defeat some of the core goals of GraphQL.
3. The use cases described here have existing solutions that are far more powerful (introspection queries, custom scalars, etc).
4. Most importantly: this issue and comment thread are vague and conflicting in what is actually being proposed.

If you or anyone have a concrete idea for how this could potentially be represented in the spec, please open a PR! My suspiscion is that the exercise will quickly reveal that such a feature is non-trivial to specify and comes with serious consequences for the primary goals of this project. But I could certainly be wrong: it's impossible to know without a clear proposal by someone who actually wants this feature.

[ericvergnaud]

@mike-marcacci
I guess it's ok to disagree with your summary?
1 - not straightforward, sure but already done, see https://github.com/timqian/gql-generator
2 - it would only defeat the goals for clients using it, certainly not for clients not using it, and not for servers who have the requirement of returning all the data when requested to do so
3 - none of the mentioned solutions have the same kind of power that '*' has. More different power is not more power.
4. Seriously ? To me the issue 100% crystal clear
Just because the primary goal of GraphQL was to NOT support such broadly specified query is not a good reason to NOT consider it now. Technology is not a religion. The ability to specify the result is a unique feature of GraphQL, but supporting wildcards does not diminish that feature, it just makes it optional.

[acjay]

@ericvergnaud -- respectfully, go ahead and make a PR for the version of this that you think should exist. Perhaps you have an approach that differs from ones that have been considered, and people may find that compelling.

Unsubscribing, as 5 years has been long enough to follow this.

[jdspugh]

I've commented on this before and other tools I've used that support wildcards that I'm in favour of. In my opinion, GraphQL is awesome apart from this missing feature. The latest development tool I've been using is the open source DADI API (https://docs.dadi.cloud/api/5.0). It's also super easy to use and setup for creating APIs and supports fine grained roles, permissions and validation. It also support wildcards through the composition parameter (https://docs.dadi.cloud/api/5.0#enabling-composition). Use https://...?compose=all to retrieve all records, their children, and their children's children, etc. My frontend, written in vanilla javascript, ends up being super compact and efficient with this powerful feature. The DADI backend is also NoSQL so you get the same schema flexibility as you do with GraphQL. I'll be using this until GraphQL supports wildcards 😉.

[murraybauer]

@acjay The comment by @ericnakagawa to discuss the issue at hand further is valid. Blocking a discussion on an issue thread by requiring a PR is probably not best.

[murraybauer]

My app requires offline syncing and without native wildcards GraphQL is a nightmare for deeply nested entities with NoSQL (so really any entity). We had to go back to REST.

I will note a new library Genql which has a ...everything sync. Yet to try. The local javascript SPA client needs a generated copy of the schema via build step to build the queries dynamically so there are trade offs.

[sungam3r]

Blocking a discussion on an issue thread by requiring a PR is probably not best.

There will be no any progress without pr. 5 years have passed. This clearly indicates the complexity of the problem.

[wildone]

Dgrapth supports expand(_all_)

[jdspugh]

There will be no any progress without pr. 5 years have passed. This clearly indicates the complexity of the problem.

Complexity? No. This problem has been solved before and has existing implementations on other platforms as has been pointed out. Now we're trying to get it to be a part of an open standard.

[sungam3r]

@jdspugh Very well. So do you mind to make initial PR into the spec repo?

[ericvergnaud]

Prior to submitting this PR, maybe it would be useful to reach consensus on the high level PR content?

1. scope
   the wildcard syntax is a client-side only specification:

• a wildcard is simply a mean to automatically convert from an 'implicit' type to an 'explicit' one. This can be achieved by the client without any help from the server (except retrieving the schema itself, which is already supported)
• a server is already bound to return all data if explicitly asked to do so, so no change required on the server to fully support the ask
• it clarifies if needed the responsibility of the server, it continues returning anything that is specified in the schema but only what is specified

2. output only
   the wildcard syntax only applies to output types, whether originating from query or mutation. No wildcard on input types (if that is even possible)

3. depth
   there seems to be consensus that specifying depth would be useful, not only to protect servers from unnecessarily deep queries but also let clients tune wildcard behavior:

• scalars (fields and lists) are included automatically
• objects (fields and lists) are only included on request
  (requires a syntax)

4. recursion and cyclic dependency
   maybe it's worth reminding that the recursion problem is NOT a wildcard only problem, as demonstrated by the number of results you get from googling 'graphql recursion' (525,000)
   I would suggest blocking wildcard recursion altogether, as follows:

• a wildcard generated output type cannot contain itself either directly or indirectly
• when such scenario is encountered, the implementation must discard the 'faulty' field and generate a warning
• if applying the above rule results in an empty output type, the implementation must discard the 'faulty' field referring to the resulting empty type and generate a warning
• the above rule applies recursively. If doing so results in no output possible, the implementation must generate an error
  Any client who needs a specific level of recursion must specify it explicitly (i.e. not using wildcards)

5. syntax
   I may have missed something, but here are the proposals so far:
   _typename_ { * }
_typename_ { ...everything }
expand(_all_) (not sure how this works)
   Fwiw, we could also consider a yaml-like syntax:
   typename { << } <- copies all fields from an alias

None of the above address the depth concern.

I would suggest:

_typename_ { ? } <- returns scalars but not objects
_typename_ { * } <- return scalars and objects recursively

alternately:

_typename_ { } <- returns scalars but not objects
_typename_ { * } <- return scalars and objects recursively

or:

_typename_ { * } <- returns scalars but not objects
_typename_ { ** } <- return scalars and objects recursively

I'm personally in favor of the first one, but will happily join the majority if it helps make something happen.

I'm happy to contribute to the actual PR, and I could invest some time implementing it in the Java client.

[jdspugh]

I suggest _typename_ { *<depth> }
e.g. posts { *2 } to return everything to depth level 2
and posts { * } to return all nodes to the greatest depth.

Very long responses may need to be truncated or produce and error. This is important for web browser clients for instance. This can be a setup parameter that allows for (a) unrestricted size, (b) truncated responses no larger than a certain number of bytes along with notification that it is truncated or (c) errors to be returned if the resulting size is larger than specified.

[coyotte508]

I like the last one:

_typename_ { * } <- returns scalars but not objects
_typename_ { ** } <- return scalars and objects recursively

And to specify depth:

• _typename_ { ** (depth) } <- return scalars and objects recursively up to depth

It's the more intuitive one in my opinion, but honestly I would be fine with any of them.

[FrankSD]

Which version will this change happen?

[sungam3r]

Most likely never.

[SephReed]

@coyotte508

I think it might also be nice to have a more explicit syntax. I like your design a lot, but given the amount of energy that is saved by this, typing some extra characters wouldn't hurt. It might also make it possible to add more features in the future (if needed). Also, it'll make the code self-explanatory.

_typename_ { * } <- returns everything it can
_typename_ { *scalars } <- return scalars but not objects
_typename_ { *depth(3) } <- return everything up to a certain depth

_typename_ { *scalars(string, number) } <- at this point, just messing around... trying to come up with other ideas to test that the spec has room for unexpected changes if necessary in the future

[sungam3r]

typename { * } <- returns everything it can

What means it can? Vague wording.

typename { * }
typename { *scalars }
typename { *depth(3) }

Even if we forget all the previous discussion of the complexity of implementing this, the question remains - who and why will use such queries in production? This is the good old select * from X, taken to a new level.

[SephReed]

It's probably ~ 100 lines of code for all of this. Recursive tree traversal is not hard at all. Command line arguments are not hard either. This is very literally CS 101 level code, and a small amount of it.

If you don't philosophically agree with the feature idea, that's one thing. But when you act like it's hard or overly complex, you are gaslighting us. You are using your position of authority to try to imply that we're crazy for thinking this is a simple possibility, but I can guarantee you this: parsing argument from a string is not hard, traversing a tree to a certain depth is not hard, and conditionally checking for nodes of a certain type is not hard.

~100 lines of code that could probably be written by a first year developer

[sungam3r]

This is all great, but you haven't answered the question. The danger of your words is that you still think in terms of one concrete implementation (maybe your own, I don't know), and not the specification, which absolutely all implementations must comply with without exception. Above in this thread, reasons have already been given why this will not work, or will work but at the cost of completely ignoring the entire GraphQL design (these are not only my answers but also from others GraphQL contributors). It’s as if you just don’t see what was being explained to you. Or don't want to see.

This is not at all about 100 (200/1000/any) lines of code that someone can write. It's about design, language specification, rules, and backward compatibility. So far, it all comes down to the fact that every time someone says "hey, it's that simple, but let's do it, why not?" but further, when counter questions arise, no one wants to think and work beyond this step.

You are using your position of authority to try to imply that we're crazy for thinking this is a simple possibility

No. If I use my position of authority (although I doubt it can count as such, there are many people who know more about spec than I do) it is only to help you.

[mike-marcacci]

Can we get this issue locked? I think that anything useful that can come of this conversation has already been discussed here, and the useful bits continue to be buried by questions and statements that have already been addressed.

[sungam3r]

Well said! I agree. I wanted to suggest it myself, but I was ashamed and did not.

[ericvergnaud]

@sungam3r @mike-marcacci How about you look at numbers?
17 voters for converting this into a PR, 3 for closing the thread...
the current situation is trying to reach consensus on the syntax.

as the conversation matures we will definitely need volunteers to go enhance existing implementations ( I agree that without such volunteering effort there is no point in this thread )

[mike-marcacci]

Hey @ericvergnaud, I'm just suggesting that this conversation as it exists here has a very low chance of being useful to anybody. I may be incorrect, but I sure struggle to see how a growing tail of poorly considered, "why hasn't this been implemented yet, it's SO easy" type comments can possibly help anyone, including their author.

For the sake of making any kind of progress, more concrete proposals (such as yours above) might instead be moved to their own issues or PRs where their specific merits and problems can be discussed without the baggage and distractions of this long, meandering, and occasionally quarrelsome thread.

Of course, if folks here do find value in rehashing previously addressed vague claims and assertions ad nauseam, that's totally fine by me, and I will unsubscribe from it. However, I do worry about leaving an issue open to comments but ignored by most folks who have deeper familiarity with the project, as any new insightful point, helpful comment, or honest question may not find an audience. And that would be sad, since many of us stay subscribed specifically so those don't end up missed.

[sungam3r]

How about you look at numbers?
17 voters for converting this into a PR, 3 for closing the thread...

This is an example of stupid 'democracy'. These numbers don't say anything. Obviously, there are far fewer people (by orders of magnitude) who are actually developing the GraphQL specification and implementation libs than people who just dropped in here to look for their own interest and liked or disliked. As I said, 17 voices say "hey, it's that simple, but let's do it, why not?" but none of them took the next step, no one wants to think and work beyond this step.

@leebyron @IvanGoncharov Indeed, please lock this issue.

[ericvergnaud]

@sungam3r it has nothing to do with democracy, it has to do with whether or not there is a strong ask or not
As a matter of fact I am proposing to take the next step, and taking time to make sure it's the correct one, so your agressive comment is ungrounded

[vraamark]

I don't think there is a strong ask. People who don't need this, usually don't drop in and say that they are all happy with the current feature set. If everything is OK in their day-to-day life, they usually have other things to do than give their votes in here.

Hot Chocolate GraphQL implementation has more than 640.000 downloads (I know that doesn't directly translate to unique users). Does that mean that 639.983 are happy and only 17 is unhappy?

There are other ways to solve the problem without change the spec, and I strongly think what is proposed here is WRONG.

[sungam3r]

I don't think there is a strong ask.

Totally agree with you @vraamark. Well said. I can say the same about GraphQL.NET - 3.4M downloads and I have not yet seen anyone suffer from lack of wildcards, at least in the last 2 years participating in the support of this and a number of other GraphQL related projects.

There are other ways to solve the problem without change the spec

Indeed. There were excellent comments that this is solved with the help of tooling. Especially if the task comes down to the knowledge of newcomers with technology.

what is proposed here is WRONG

But not everyone can understand this 😕

[IvanGoncharov]

Locked this issue as suggested above ⬆️