GET http://component-crawler.herokuapp.com/.json returns empty dataset

[xmojmr]

..with this content

{"users":[],"components":[]}

Did the API change? Is it temporary service unavailability? Anything I can do to "fix it" on my side?

[timaschew]

so http://component.github.io/ and http://component.xmojmr.cz/ are affected

Did the API change?

No

I tried out to add some data via http://component-crawler.herokuapp.com/ and then it worked for a while, but at the end it become the same problem again. So it looks like it cannot persist the data anymore.

The heroku app shows some env variables for s3 like
S3_BUCKET, S3_KEY and S3_SECRET but the app was not created and configured by me.

@jonathanong can you help out? Did you change something in the last days?

[jonathanong]

ohhhhh i was cleaning up my IAM credentials. do you guys mind using a new s3 bucket? i don't have control over the app anymore so i can't change them

[timaschew]

I just created a AWS account.
I'm not familiar with AWS. So I created a bucket and a user via IAM with a full access to the S3 service.
Do I need something else? Or can you transfer something from your account? my username is [email protected]

[jonathanong]

just create a bucket and set the environment variables in heroku. should be all you need.

[timaschew]

okay, but I'm not sure if it's working.
If I restart the app, the data is empty again.
If I don't restart the app and just wait some time it's working and I think this didn't work before.
// it still doesn't work

Logs via heroku only show some errors for the /logs resource, but not for /.json.

[vko-online]

it still doesn't work

[timaschew]

yep, I know. I will investigate the issue on wednesday again

[timaschew]

for now the old wiki can be used: https://github.com/componentjs/component/wiki/Components

[vko-online]

Why this is better than bower? What's the advantages at least

[timaschew]

bower is just a package manager, component is both: a package manager and a build tool, so it provides require implementation and assets linking for instance for css, fonts, etc.

[vko-online]

It's like bower + gulp? Or bower + browserify? Sorry but i don't get the idea, what's the concept

[sankargorthi]

https://github.com/componentjs/guide This should get you started. Also, componentjs/component#594 (comment)

[timaschew]

It's like bower + gulp? Or bower + browserify?

it's more like npm + browserify, but it uses github instead of npm.

If you want to see how it works and how to use it, you should try it some of these examples: https://github.com/timaschew/c8-experiments

or read the guide

[timaschew]

fixed now, I added components of the component organisation, but we need to get all the data back.

@jonathanong do you have a backup / still the file in your bucket db.json ?

[xmojmr]

@timaschew I have some ~2 months old backup of already filtered data but with the list of components known at that time.

You can get it from http://component.xmojmr.cz/api/v1/crawler.json

However, once you update the real crawler, the above link will be invalid. There's a CRON job running at my site doing refresh every two hours

[jonathanong]

https://s3.amazonaws.com/component_crawler/db.json

[jonathanong]

Last Modified: Sat Mar 28 14:17:21 GMT-700 2015

[timaschew]

thanks!

I uploaded jonathan's json, because it had a little bit more components (2813) instead of xmojmr's with 2791 components

[xmojmr]

@timaschew thanks a lot, my site is up again as well.

As one more follow up action it might be useful to write down the steps needed to install/modify this service. At least from my point of view, the process is not obvious even after reading source code and @jonathanong's comments.

But it's more-less pointless as long as you don't abandon the ship 😉

[timaschew]

@xmojmr I fixed some issue and updated the readme, you can try it now on your local machine.