[Feature Request] CREST Pilot Tracking

[vivace-io]

This falls in line with #83 which is asking for SSO. This could be extended further to use the new CREST endpoint, detailed in this dev blog. The advantage of this would be that the users do not need to keep the IGB open to track their location, and the cache time is a short 10 seconds. Thoughts? I'm hoping that once I know my way around I can help with this :)

[exodus4d]

Yeah it is on my "TODO" list... ;)

[exodus4d]

So this is currently in work.

Done so far:

• Added Login process over CCP´s SSO.
• Added oAuth2.0 functions for all relevant basic actions (getAccessToken(), refreshAccessToken(), walkEndpoint(), showEndpoints(),...)
• Added refresh function that automatically refreshes" online" characters and verify their position/system and some other data.

There are some design decisions to made now. Im not 100% sure how what is the best way to do some things. The current (online) state:

• Each user can have infinite API Keys,
• Each API Key can handle up to 3 Characters
• Each "character" owned by a "user" can access all the privat maps from that user. This means, "private" maps are bound to a user, and not to a character. The advantage of this is, that all "alts" can edit/see all of your maps.

With the CREST API, some things will change:

• It is not possible to "grap" all characters from a user (person). I can just access the single character who logged on. Therefore a person will not be able to "see" all private maps of all his characters!
• This leads us to the fact, that "privat" maps can not be "bind" to a "user" (as present), but to a "character".

CCPs SSO does not provide a way to log in multiple characters at once. The only way to "ship around" that, is to still create "users" on Pathfinder and let them manually log in multiple characters, which is not very user friendly.

[Zumochi]

Perhaps it would be possible to register a user, and then link characters to said user by logging in through SSO.

Users who don't want an account can stick to just using SSO.

[vivace-io]

Does CREST destroy the old token or disable the token from being refreshed if you attempted to authorize a second character on the same account? If it does not, you could do basically what Zumochi said and have SSO for authentication and then a second "layer" allowing them to log in the individual characters after which they will not have to do it again.

EDIT: Also, can you push this work somewhere so I can take a loot at it?

[exodus4d]

No CCP does not "destroy" any valid access_key. The only thing I have to take care of, is to "refresh" any access_key when the 20min expire time expires.
So in theory it is possible to log in a user, and then refresh the "access_key" by its "refresh_token" until your server or CCPs API shuts down for more than 20min (which can take month).

The "second layer" is probably the easiest solution for now. If there is no "user" created and associated with a character, Ill create a new user. If there is already a user logged in, Ill connect the new character to that user.... There is still one problem: If the CREST API is offline, no one can access or edit the maps. Ill not provide multiple ways of logging in (with API, without API) that is for sure :)

Ill push my work to the public develop branch if there is a stable state available which can take some weeks.

[Zumochi]

There'll be more issues when the CREST API is offline, so that'd be an entirely understandable time for the mapping tool to be unavailable as well. In that case, the user can always fall back to using IGB until that is removed as well.

Maybe a feature branch separate from develop can be created btw?

[exodus4d]

New feature branch is now publicly available "Branch #84". It is still a very early WIP state and can not be used in "production". But If you are intrested in checking the code or implementation. feel free... :)

Some required information for testing:

1. Register your App by CCP to get API access ( Client_ID and Secret_key) which is required to use the (not private) CREST API (UR: https://developers.eveonline.com/applications) (check screenshot)
2. Add the required CREST "Scopes" you want to have access (check screenshot)
3. Add the callback URL (check screenshot)

4. Add your Client_ID and Secret_key data to environment.ini
5. Thats it!

How it works

• All the "magic" happens in here: ccpssocontroller.php
• There is a small "SSO login" link next to the login form login.html L148
• When clicking the link you will get redirected to CCP s Auth-URL (HTTP 302) (see official docu.)
• Login with your Acc (the required Scopes set up for your App can be seen):
  
• After login, your App should get a a valid access_token and refresh_token whith will be used from now on for any API request.
• The access_token is valid for 20min (CCP default). I store them in session (remember -> session data is saved to DB table session) with a ttl of 1200s (20min)
• All the API caching is done by the framework itself (by there Cache-Control response Header). But i found a bug in F3 which is not yet fixed "Issue 908"
  All CREST API paths are "dynamic" and are fetched automatically by recursive API calls "walkEndpoint(); L309" (which corresponds to the official documentation). As an example of that, check out the getCharacterData(); function which "walks" down the CREST API root path, until the requested "leaf" is reached and returns the data "L390"
• All the JSON date coming from the CREST API is mapped by a OOP implementation of _PHP_s "RecursiveArrayIterator()" class (checkout: mapper classes )
• Some "static" tables are automaticalls filled/updated (characte, corporation, alliance)...

With the current state we can access any CREST API data after successful login by a very generic CREST controller. All the caching is done automatically by the framework and we don´t have to care about it).

Next steps

• Code a nice intuitive UI for SSO login
• "Bringing everything together" :) Make sure all the CREST data is "in sync" with the DB, update, add, remove location data for logged in characters.....
• This will massively reduce the AJAX response timings which are done by each client to update the map data for each client.
• In addition to that, i have added a second DB-Server which is running a database replication of MariaDB for "read* queries. This is of course not required... and was just a test with a huge success. -> The DB response time could be reduced by ~ 150% !

Feel free to comment your ideas or any suggestions for improvement. Maybe we could arrange a Teamspeak talk or something like that...

---

Official CREST Docu: http://eveonline-third-party-documentation.readthedocs.org/en/latest/sso/intro/

[VivianMeally]

Can it be allready used? Or is it only for testing? Dont see this SSO Login login at https://www.pathfinder-w.space/

[Tupsi]

Trying out branch #84 atm and got the whole auth process working, but instead of a "real login" I only get a bunch of text back after authorization, in the form of

array(10) { ["created"]=> string(19) "2016-03-11 10:45:18" ["updated"]=> string(19) "2016-03-11 10:45:18" ["active"]=> bool(true) ["name"]=> string(6) "McTest" ["corporationId"]=> array(9) { ["created"]=> string(19) "2016-03-10 18:44:28" ["updated"]=> string(19) "2016-03-10 18:44:28" ["active"]=> bool(true) ["name"]=> string(25) "Dumbo Corp" ["shared"]=> bool(false) ["isNPC"]=> bool(false) ["corporationCharacters"]=> NULL ["mapCorpo... and so on.

but not a working pathfinder behind it. Can only hit reload which gets me back to the login screen.

Is this an intended behavior currently, because its just basic sso testing, or should this already be going further into the app?

[Tupsi]

please see my issue #147 I just opened. Seems you need to escape the names you get from ccp a bit more.

[exodus4d]

#84 branch is deprecated and will be removed soon. Developer branch is
up2date and a working deployment of it can be found at
https://www.dev.pathfinder-w.space .this is a new Test Server for the
upcoming CREST Release.
I'll check the escaping too. Thanks

Ugurtan [email protected] schrieb am Fr., 13. Mai 2016, 15:58:

please see my issue #147
#147 I just opened. Seems
you need to escape the names you get from ccp a bit more.

—
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHub
#84 (comment)

[Tupsi]

thanks, and you can ignore my old remark to #84 as I am already using your dev branch.

[Tupsi]

the nginx error log is filling up with lots of messages like this one and the result is, that the user creating these is hit with a server error message in the browser and needs to reload the pathfinder website.

2016/05/14 15:30:59 [error] 11392#11392: *344797 FastCGI sent in stderr: "PHP message: method: 'GET', url: 'https://crest-tq.eveonline.com/characters/[characterID]/location/'" while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: myserver.net, request: "POST /api/map/updateUserData HTTP/2.0", upstream: "fastcgi://127.0.0.1:9000", host: "myserver.net", referrer: "https://myserver.net/map"

[Tupsi]

here is the one leading to a 500 server error with an "emergency shutdown" from pathfinder.

2016/05/15 19:20:52 [error] 3459#3459: *804671 FastCGI sent in stderr: "PHP message: method: 'GET', url: 'https://crest-tq.eveonline.com/characters/366150322/location/'
PHP message: [app/main/lib/web.php:128] Base->error(500,'method: 'GET', url: 'https://crest-tq.eveonline.com/characters/366150322/location/'')
PHP message: [app/main/controller/ccp/sso.php:516] Lib\Web->request('https://crest-tq.eveonline.com/characters/366150322/location/',array('timeout'=>3,'method'=>'GET','user_agent'=>'PATHFINDER - v1.0.0RC3 | https://github.com/exodus4d (pathfinder.tdsin.net)','header'=>array('Authorization: Bearer 63Wcfxxxxxxxg97eUth9tfbsqYVtwia_KKmRnFHkV2-c-dI0xxxxxx3SpPi-75yP3SxxxxxxxVgJUiqMYw2','Host: login.eveonline.com','Host: crest-tq.eveonline.com')),array('suppressTimeoutErrors'=>true))
PHP message: [app/main/controller/ccp/sso.php:555] Controller\Ccp\Sso->getEndpoint('63Wcfp0Qxxxxxxia_KKmRnFHkV2-c-dI0t0qIyyyyySpPi-75yP3SSmP5ucswGVVgJUiqMYw2','https://crest-tq.eveonline.com/characters/366150322/location/',array('suppressTimeoutErrors'=>true))
PHP message: [app/main/controller/ccp/sso.php:556] Controller\Ccp\Sso->walkEndpoint('63Wcfp0xxxxxxUth9tfbsqYVtwia_KKmRnFHkV2-c-dI0t0qxxxxxSpPi-xxxxxxxxxxUiqMYw2',array('standings'=>array('href'=>'https://crest-tq.eveonline.com/standings/366150322/'),'bloodLine'=>array('href'=>'https://crest-tq.eveonline.com/bloodlines/1/','id'=>1,'id_str'=>'1'),'gender_str'=>'1','waypoints'=>array('href'=>'https://crest-tq.eveonline.com/characters/366150322/navigation/waypoints/'),'private'=>array('href'=>'https://crest-tq.eveonline.com/characters/366150322/private/'),'channels'=>array('href'=>'https://crest-tq.eveonline.com/characters/366150322/chat/channels/'),'href'=>'https://crest-tq.eveonline.com/characters/366150322/','accounts'=>array('href'=>'https://crest-tq.eveonline.com/accounts/366150322/'),'portrait'=>array('32x32'=>array('href'=>'http://imageserver.eveonline.com/Character/366150322_32.jpg'),'64x64'=>array('href'=>'http://imageserver.eveonline.com/Char

[Dessix]

This seems to be what I'm getting, as mentioned in #156.

[exodus4d]

Issue resolved in master branch