Skip to content

๐Ÿ“ฆโœ‚๏ธ๐Ÿ“‹๐Ÿ“ฆ Create a mirror of packagist.org metadata for use locally with composer

License

Notifications You must be signed in to change notification settings

vrkansagara/packagist-mirror

ย 
ย 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿ“ฆ Packagist Mirror

Build Status docker Status docker pulls Minimum PHP Version Packagist Codecov Quality Score Mentioned in Awesome composer

This repository make possible to you create a metadata mirror and keep in sync with packagist.org.

Mirror creation

โค๏ธ Recommended by packagist.org โค๏ธ

If you're using PHP Composer, commands like create-project, require, update, remove are often used. When those commands are executed, Composer will download information from the packages that are needed also from dependent packages. The number of json files downloaded depends on the complexity of the packages which are going to be used. The further you are from the location of the packagist.org server, the more time is needed to download json files. By using mirror, it will help save the time for downloading because the server location is closer.

โš™๏ธ How it works?

This project aims to create a local mirror with ease, allowing greater availability for companies/countries that want to use the composer but do not want to depend on the infrastructure of third parties. It is also possible to create a public mirror to reduce the load on the main repository and allow a better distribution of requests around the world, help us creating a public mirror!

When creating your local copy the child mirror (you) adds a list of other mirrors to use when creating your own mirror, if any mirror fails to deliver the metadata file the client automatically fetches the original file from the main mirror, could be packagist.org or even another. If you lose your connection to the server or any other problem that prevents you from continuing is okay, it can return from where it stopped running. After the mirror is created, the next runs will only look for the main mirror delta optimally and efficiently.

Mirror creation

๐ŸŒŽ Packagist public metadata mirrors observatory around the world

๐Ÿ›ซ Amazing data mirrors used to download repositories metadata built using this recommended repository or another:

Location Mirror Maintainer Github Sync Since
India packagist.vrkansagara.in Vallabh Kansagara fork Every 5 minutes Q4'19
Brazil packagist.com.br Webysther main Continuously Q3'17
China php.cnpkg.org Eagle Wu fork Every minute Q3'18
China mirrors.aliyun.com Aliyun Every 5 minutes
China packagist.mirrors.sjtug.sjtu.edu.cn Shanghai Jiao Tong University fork Every hour Q2'19
China mirrors.cloud.tencent.com Tecent Cloud Every day
Indonesia packagist.phpindonesia.id Indra Gunawan fork Every 30 seconds Q3'18
Indonesia packagist.ianmustafa.com Ian Mustafa fork Every 30 seconds Q3'19
Japan packagist.jp Hiraku forked Every 2 minutes Q4'14
Japan packagist.kawax.biz Kawax another Every hour Q4'18
South Africa packagist.co.za SolidWorx fork Every 5 minutes Q3'18
South Korea packagist.kr PackagistKR fork Every minute Q3'18
Thailand packagist.mycools.in.th Jarak Kritkiattisak fork Every 5 minutes ๐Ÿ†•Q4'19

Not working as a mirror of packagist.org (checked at Q4'19):

Location Mirror Maintainer Github Reason At least Since
China mirrors.huaweicloud.com Huawei Cloud Outdated Q3'19
packagist-mirror.wmflabs.org Wikimedia fork Outdated Q3'19 Q3'18
India packagist.in Varun Sridharan fork HTTP2 PROTOCOL ERROR โš ๏ธQ4'19 Q2'19
China packagist.phpcomposer.com Outdated โš ๏ธQ4'19
China nas.tbpark.org ่ตตไธฐ (Zhao Feng) fork CONNECTION TIMED OUT โš ๏ธQ4'19 ๐Ÿ†•Q4'19

If you know any new mirror based or not on this one, please create a issue or a pull request with the new data.

Check status page for health mirror's.

World Map

The colors represent the topology drawn below, only show servers in better condition at country level.

๐Ÿš€ Create your own mirror

Topology

With docker and nginx:

The mirror creation save all data as .gz to save disk space and CPU, you need to enable reverse gz decode when a client ask for the decompressed version, normally used only for legacy composer clients.

Change you nginx configuration of gzip_static and gunzip as is:

Create a website on a default nginx instalattion:

sudo vim /etc/nginx/sites-available/packagist.com.br.conf
server {
    index.html;

    server_name packagist.com.br www.packagist.com.br;

    location / {
        try_files $uri $uri/ =404;
        gzip_static on;
        gunzip on;
    }
}

๐Ÿ’กTip: use a machine with 2GB at least of memory, with that all metadata keep to the memory helping the nginx and disk to not be consumed at all.

After install nginx edit /etc/crontab:

* * * * * root docker run --name mirror --rm -v /var/www:/public \
-e MAINTAINER_REPO='packagist.com.br' \
-e APP_COUNTRY_NAME='Brazil' \
-e APP_COUNTRY_NAME='br' \
-e MAINTAINER_MIRROR='Webysther' \
-e MAINTAINER_PROFILE='https://github.com/Webysther' \
-e MAINTAINER_REPO='https://github.com/Webysther/packagist-mirror' \
-e URL='packagist.com.br' \
webysther/packagist-mirror

to more options about image go to docker repository.

Put inside your ~/.*rc (~/.bashrc/~/.zshrc/~/.config/fish/config.fish):

alias logs='watch -n 0.5 docker logs --tail 10 -t mirror'

Update your env vars and see monitoring packagist mirror creation:

source ~/.*rc
logs

๐Ÿ‘ท Install

Using with docker repository or composer local:

$ git clone https://github.com/Webysther/packagist-mirror.git
$ cd packagist-mirror && composer install
$ cp .env.example .env

Schedule the command to create and update the mirror:

$ php bin/mirror create -vvv

๐Ÿง Development & Contributing

Please see CONTRIBUTING and CONDUCT for details.

๐Ÿ›ฃ๏ธ Roadmap

  • Translate readme.md and index of mirror.
  • More recipes to AWS/Azure/GCP/DigitalOcean and another cloud providers.
  • Support gz disabled for limited configuration access to Apache/Nginx.
  • Support full mirror mode (for countries/companies with limited access to internet).
  • Status report for all mirrors around the world.
  • Stats after initial mirror creation.

๐Ÿ“‹ Requirements

The following versions of PHP are supported by this version.

  • PHP >=7.2

๐Ÿงช Testing

$ vendor/bin/phpunit

๐Ÿฅ‚ Credits

๐Ÿ’™ Other correlated projects

  • Zencodex Create a mirror
  • IndraGunawan Create a mirror (outdated)
  • Hirak Create a mirror (outdated)
  • Slince Easily switch to the composer repository you want
  • Packagist Bot Yet Another Packagist Mirror
  • Twity Provide a web based management of private and public composer packages.
  • Velocita Proxy Composer caching reverse proxy

โ˜ฎ๏ธ License

MIT License. Please see License File for more information.

FOSSA Status

About

๐Ÿ“ฆโœ‚๏ธ๐Ÿ“‹๐Ÿ“ฆ Create a mirror of packagist.org metadata for use locally with composer

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • PHP 100.0%