Couchdb support

[BitPuffin]

Hey, it would be really awesome if you added couchdb support, it's really one of the only things keeping me away from vibe.d!

I could try and hack on it myself I guess, could be fun to take on :)

[s-ludwig]

That would be great, of course, if you would give it a shot ;)

I'm far too busy with the existing functions and other stuff to tackle this within the coming months. I guess there is no existing D implementation of a couchdb driver? In that case a port would probably be pretty straight forward - but it seems like the protocol is quite simple to implement, maybe even utilizing the vibe.http.rest module... grr now it sounds like a fun project but things like a full win32 back end are waiting in line and then epoll and winrt and DoS protection and...

[BitPuffin]

Sounds like I am going to have to take it on then if I want to use Vibe (it would be pretty bad ass). And the Couch API is really just a bunch of HTTP stuff and I need to learn Couch better so why not :)

[BitPuffin]

I'm trying to use the rest module from vibe to generate an interface, however it's a bit tricky.

CouchDB's REST API goes kind of deep sometimes, and sometimes the url's require the use of underscores.

For example:
GET /_config/section/key
or
POST /db/_design/design-doc/_list/list-name/other-design-doc/view-name

Is there any way that I can match those patterns with interfaces for RestInterfaceClient?

I'm getting some ideas to perhaps wrap things in classes, it would be nice if you could point me in the right direction!

[s-ludwig]

Right now that would unfortunately require a class for each level, somehow like this, with MethodStyle.Unaltered:

class CDB { @property DB db() { return _db; } }
class DB { List get_list(string id) { return _databases[id].list; } }
class List { Doc get(string id) { return _docs[id]; } }
class Doc { Entry get(string id) { return _entries[id]; } }

Currently the magical id parameter is the only way to dynamically match parts of the URL and it is also not possible to a whole path dynamically (e.g. to match any sub path and get back the path as a string[]). But now with user defined attributes it could be made much more flexible, e.g.:

class DB { @restName!"_list" List getList(string id) { return _databases[id].list; } }
class List { Entry get(@pathMatch!0 string list, @pathMatch!1 string entry) { return _docs[list][entry]; }

But I didn't have the time to implement or even think about a design for this, yet. If you have any good ideas for a clean way to annotate all such things (or even would like to implement it), that would be good. But I hope to have some time for this within the next month if nothing gets in-between.

[s-ludwig]

Hmm, and with class I meant interface, since this is for a client...

[BitPuffin]

Well what initally seemed natural to me would be something like this:

interface ClientAPI {
    @property ConfigAPI _config(); 
    Json get(); // GET /
    DbAPI getDb(string db);
}

interface ConfigAPI() {
    Json get(); // GET /_config
    Json get(string section); // GET /_config/section
    Json get(string section, string key); // GET /_config/section/key  -- not sure if it's a good way
    void put(string section, string key, Json value); // PUT /_config/section/key with value as the body and content-type application/json
    Json delete(string section, string key); // DELETE /_config/section/key
}

interface DbAPI {
    @property Json _changen();
    // Here is where it falls apart: POST /db/_compact whick doesn't have a body, and the _ makes it ideal for a property
    // And how would I cleanly implement GET/PUT /db/_revs_limit
}

But as you see it eventually dies on me.

I guess what could be useful would be if there was a way to annotate what http method to use.

And the real problem is that afaic there is no quick and easy way to wrap a REST api manually. With my current WIP implementation (https://github.com/BitPuffin/DouchVibe) I use HttpClient which is sort of "low level" for something as simple as this and right now the result is pretty messy.

What would be pretty sweet would be if there was a kind of UrlRouter like utility for manually mapping a REST API to specific functions. That way we wouldn't have any trouble knowing if the intention is to use GET or POST or PUT or.....

Thoughts?

[mihails-strasuns]

AFAIK,

Json get(string section);

is supposed to be either

Json getSection();

if it is meant to be exactly "section"string or

Json get(string id);

if it is meant to be GET /:section-name . But if multiple variables are nested, things get nasty.

@s-ludwig , if you have some time to write a spec for more advanced variable handling in REST urls, I can implement it for you, right in the middle of choosing "next important thing to code".

[BitPuffin]

@Dicebot Well section is actually not in the url, it's like GET /_config/:section

Also I've ran into another issue, CouchDB uses an almost REST thing where it really only uses the rest it likes. So couch requires support for the HTTP method COPY which isn't in the HttpMethod enum. How should I work my way around this?

[BitPuffin]

Maybe the HttpMethod needs to be updated. HTTP is ment to be extendable as far as I know, so maybe changing it so that it's more generic somehow is a good idea?

[mihails-strasuns]

@BitPuffin I'll try to address your issues once Sonke's opinion on how final design should look like is read, as he have said he himself is a bit busy now. Let's do some open-source friendship magic here :)

[BitPuffin]

@Dicebot Nice! I'll continue finishing the CouchDB plugin when this is addressed. But I might have to (temporarily at least) write the app I'm working on in a different language and then port it to Vibe later on.

[s-ludwig]

Regarding HTTP methods, I would like to try and see how a pure enum approach suffices. Originally, the method was just represented as a string, but for performance reasons it was replaced by an enum. A hybrid approach could be used to only trade performance in case of unknown methods, but lets see first if it is enough to just add all well defined methods to the enum.

Regarding the full customization of certain interface methods, I've thought that maybe the following two annotations are a cleaner approach than the per-parameter annotation and should be more flexible:

1. @path("/_config/:section/:key") void getConfigKey(string _section, string _key);
2. @method(HttpMethod.COPY) void copyEntry(...);
3. If the UrlRouter would also store the "_" match as a param, something like this would also work: @path("/some/path/_") void method(string _tail);or possibly@path("/some/path/*") void method(Path _tail);

The @path would get the path of the parent interface prepended (if any) and then passed to the UrlRouter as-is. Since the UrlRouter adds a param to the request for each match, the rest should already work fine. @method would just manually set a HTTP method and use the full function name (no verb stripped) as the base for the generated URL (if no override using @path is given). [The whole _pname thing could of course also replaced now by something along the lines of void method(@param("pname") my_param);]

For sure it would be great if you would get this started, @Dicebot. I'm currently feeling like a tornado of tasks is whirling around me :) Any comments/improvements are of course very appreciated. Also, the REST interface generator is growing quite impressively, a bit more and we need a range of tutorials to cover all functionality ;)

BTW, the initial intention for the REST interface generator was more to have a tool to quickly define a REST interface rather than implementing an existing definition. That's the main reason why the current functionality doesn't feel quite comfortable for this kind of application

[BitPuffin]

@s-ludwig In this implementation, would it be possible to annotate with both @path and @method?

[s-ludwig]

Yes, @path would just override any effect on the path, but the changed HTTP method would still apply.

[BitPuffin]

Well I mean at the same time

[BitPuffin]

If not I think there shold be a combo annotation! (The reason I'm asking is because I haven't learned about D function tagging yet)

[s-ludwig]

Yes, at the same time:
@method(HttpMethod.POST) @path("/my/path") void getItOn();
shoud work, as well as:
@(method(HttpMethod.Post), path("/my/path")) void getItOn();

Both would mean that getItOn() will me available as POST /my/path instead of GET /it_on.

[s-ludwig]

Documentation for UDAs: http://dlang.org/attribute.html#uda
(that language reference is really not good for finding specific things... I had to use the search engine to actually find the right spot)

[BitPuffin]

Okay! Perfect then, I hope this gets implemented soon enough so I can use D for my project with the deadline of thursday next week.

In the mean time I'll have a look at making a scrypt dub package.

[mihails-strasuns]

@s-ludwig Sounds solid, will start hacking this tomorrow. I have a few concerns about proposed implementation but I guess need to check current one first.

@BitPuffin No deadline promises, sorry, it will go as it goes.

[BitPuffin]

@Dicebot Well I didn't mean to imply any pressure on you. I was really just thinking in general, that if this gets implemented quickly I'll finish the project in D, if not I'll just do it in ruby or something!

[mihails-strasuns]

@BitPuffin Please follow #189