Every web site provides APIs.
Toapi is a clever, simple and fast library letting any web site provide APIs. In the past, we crawl data and storage them and create api service to share them maybe we should also update them regularly. This library make things easy. The only thing you should do is defining your data, they would be shared as api service automatically.
- Documentation: http://www.toapi.org
- Awesome: https://github.com/toapi/awesome-toapi
- Organization: https://github.com/toapi
from toapi import XPath, Item, Api
from toapi import Settings
class MySettings(Settings):
web = {
"with_ajax": False
}
api = Api('https://news.ycombinator.com/', settings=MySettings)
class Post(Item):
url = XPath('//a[@class="storylink"]/@href')
title = XPath('//a[@class="storylink"]/text()')
class Meta:
source = XPath('//tr[@class="athing"]')
route = {'/news?page=:page':'/news?p=:page'}
class Page(Item):
next_page = XPath('//a[@class="morelink"]/@href')
class Meta:
source = None
route = {'/news?page=:page':'/news?p=:page'}
def clean_next_page(self, next_page):
return "http://127.0.0.1:5000/" + str(next_page)
api.register(Post)
api.register(Page)
api.serve()
# Visit: http://127.0.0.1:5000/
- Sending only one request to source web site with the same url.
- Most of the data fetched from cache and storage.
- Getting HTML from storage when the cache expired.
- Getting HTML from source site when the storage expired.
$ pip install toapi
$ toapi -v
toapi, version 0.1.12
$ toapi new api
2017/12/14 09:16:54 [New project] OK Creating project directory "api"
Cloning into 'api'...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 10 (delta 1), reused 10 (delta 1), pack-reused 0
Unpacking objects: 100% (10/10), done.
Checking connectivity... done.
2017/12/14 09:16:56 [New project] OK Success!
cd api
toapi run
In the directory of 'api' created above. Run the command line as follows.
$ toapi run
2017/12/14 09:27:18 [Serving ] OK http://127.0.0.1:5000
Then, everything is done. Visit http://127.0.0.1:5000 in your browser!
A Toapi app is a flask app. So you can deploy it as follows:
While lightweight and easy to use, Flask’s built-in server is not suitable for production as it doesn’t scale well and by default serves only one request at a time. Some of the options available for properly running Flask in production are documented here.
If you want to deploy your Flask application to a WSGI server not listed here, look up the server documentation about how to use a WSGI app with it. Just remember that your Flask application object is the actual WSGI application.
Deployment Options — Flask Documentation (0.12)
$ toapi new toapi-pic
$ cd toapi-pic
$ toapi run
# http://127.0.0.1:5000/_items
{
"/pic/?q=:key": [
"Pixabay",
"Pexels"
]
}
# http://127.0.0.1:5000/_status
{
"cache_get": 2,
"cache_set": 2,
"received": 4,
"sent": 2,
"storage_get": 1,
"storage_set": 2
}
# http://127.0.0.1:5000/pic/?q=coffee
{
"Pixabay": [
{
"img": "https://cdn.pixabay.com/photo/2017/06/21/05/28/coffee-2426110__340.png"
},
{
"img": "/static/img/blank.gif"
}
],
"Pexels": [
{
"img": "https://images.pexels.com/photos/302899/pexels-photo-302899.jpeg?h=350&auto=compress&cs=tinysrgb"
},
{
"img": "https://images.pexels.com/photos/34085/pexels-photo.jpg?h=350&auto=compress&cs=tinysrgb"
}
]
}
Toapi use cache to prevent repeated parsing and use storage to prevent sending request.
A toapi app has an ability to gather pages of multiple websites and convert them to easy to use APIs
Any application created by toapi could be shared to others.
A toapi app is a standard flask app, so that you can deploy your app as deploying a flask app.
A toapi app will automatically count kinds of states of itself and you can visit the states whenever you want.
To get help with Toapi, please use the GitHub issues
- Checking system every time running the app.