Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bitcoin-Intelligent-Life - Blockchain engineering class project #4039

Closed
3 of 4 tasks
synctext opened this issue Nov 19, 2018 · 41 comments
Closed
3 of 4 tasks

Bitcoin-Intelligent-Life - Blockchain engineering class project #4039

synctext opened this issue Nov 19, 2018 · 41 comments
Assignees
@rwblokzijl @MattSkala @Jaapp- @andrebreis @rpytel1 @HusainKapadia
Labels
type: MSc course work

Comments

@synctext
Copy link
Member

synctext commented Nov 19, 2018 •
edited
Loading

Project : Bitcoin-Intelligent-Life

Self-replicating Bitcoin-based entities using deep re-enforcement learning

You will create a key prototype that advances the state-of-the-art of self-replicating software, autonomy, and artificial intelligence. Your mission is not to terminate all human life.

image

Bitcoin is an essential technology for digital life: you can buy servers. In prior work TUDelft has created CloudOmate. With CloudOmate you can buy servers with Bitcoin automatically. CloudOmate gives any computer the power to own money, replicate freely, no human can access that server, and no human can take that money away. It becomes an autonomous virtual entity. With cloudomate can even enhance it's own privacy and buy VPN networking autonomously.

Next step is adding intelligence and learning to this autonomous virtual entity. You will combine buying servers with Bitcoins with the another peak-hype technology of today: AI. You will create a basic intelligent entity, capable of self-replication and learning capability. For the learning side you can use deep re-enforcement learning engine in Python. See basic tutorial

Possible sprints for the 10 weeks:

  • get CloudOmate going and repair broken APIs to VPS providers and payment gateways
  • create the first primitive Q-table in Python with the focus on survival
  • add primitive support for decentral market trading and income earning
  • focus on source of sustainable income for the use existing Python code for Tor-like exit node services

Outcome: running code and Github readme.md (no thick final report that nobody really reads).

Warning: this is a scientifically most challenging assignment (recommended for 'Cum Laude' level students)
Supervised by: Dr. Johan Pouwelse
@synctext
Copy link
Member Author

synctext commented Nov 19, 2018 •
edited
Loading

Kick-off meeting:

@devos50
Copy link
Contributor

devos50 commented Nov 19, 2018

Please note that the 2017 plebnet project has been continued by another group this year. See the last few posts of this ticket: #2925

Report: https://repository.tudelft.nl/islandora/object/uuid%3A614efc19-e6a9-453f-8084-2e2c54ddd93f

They made this visualization:
42031541-16e66e28-7ac6-11e8-8ddd-c2db87de599f

@synctext
Copy link
Member Author

Meeting minutes:

  • time spend mostly coding, less about reading prior stuff
  • Fix BlueAngelHost MattSkala/cloudomate#7
  • documented in a single list an overview of prior work (from 2016 onwards)
    • create single table overview of prior groups
    • github, final report, issue, and cumulative student numbers
    • (no need for student names if you have GithubIDs)
  • find other providers, using the cart.php trick
  • plebnet self-replication, docker needs repairing or something is broken

Next sprint ideas:

  • focus on Plebnet operational
  • wrap up cloudomate
  • first Q-table reward function and global agent design
    • deliberately as primitive as possible
    • select between VPS providers
      • state: current VPS provider
      • action: stay or switch
    • market strategy
      • blind and static
      • do not take into account you inventory, demand or pricing fluctiations (next sprints)
      • earn Tribler tokens for whole month and sell everything in last 24h of VPS account
      • 1 offer on the market, price: sell everything at price point to buy exactly 1 new VPS
    • Tor-like Tribler exit traffic only, no seeding or credit mining yet
    • upon death: global recovery wallet

@qstokkink qstokkink added this to the Backlog milestone Nov 27, 2018
@andrebreis
Copy link

We are using this Plebnet repository

@rwblokzijl
Copy link
Member

rwblokzijl commented Dec 2, 2018 •
edited
Loading

Timeline of the PlebNet and Cloudomate Projects

This post serves to give an overview of the history of the PlebNet and Cloudomate projects. It aims to give a high level overview of the project groups that have worked on the project and their contributions. 5 teams have worked on this project before we (team 6) got started.

Team 1: TENNET (Skynet 2.0)

This is the inception of the project. The project started in april 2016 as the bachelor end project of:

In this first project a miminum working project was created. Skynet 2.0 added:

  • primitive automation of buying VPS's
  • primitive self replication to other machines using this automation given
    enough bitcoin.
  • A primitive DNA structure for genetic learning

After this project there was not working version yet as money making was not properly implemented.

Documentation:

Team 2: PlebNet

The PlebNet project started in May 2017 as the bachelor end project of:

This project improved on Skynet 2.0 by:

  • extracting the VPS buying code into a separate project called cloudomate, with restructured code, better web scraping techniques, and more VPS providers implemented.
  • expanding on the DNA structure and mutations.
  • implementing:
    • Tribler reputation mining
    • trading reputation for Bitcoin
    • buying servers with Bitcoin
    • installing itself on the servers to start mining reputation again

Documentation:

Team 3: Growing Life using Blockchain

Starting November 2017 the PlebNet project was improved upon by:

Note that Team 3 and 4 worked parallel.

The project improved upon Cloudomate by:

  • Fixing broken VPS'
  • Migrating to python3
  • Adding automated VPN purchasing

The project improved upon PlebNet by:

  • Integrating it with their improved version of Cloudomate

Documentation:

Team 4: Growing primitive blockchain life

Starting November 2017 as well, the PlebNet Project was improved upon by:

Note that Team 3 and 4 worked parallel.

The project improved upon Cloudomate by:

  • adding automated VPN purchasing
  • found a way around captchas and google ReCaptcha

The project improved upon PlebNet by:

  • adding the option to use an Etherium wallet

Documentation:

Team 5:

Starting April 2018 the PlebNet project was improved upon as the bachelor end
project of:

The project improved upon Cloudomate by:

  • fixing broken VPS'
  • adding a random user generator

The project improved upon PlebNet by:

  • dynamically checking what VPS providers cloudomate has access to at the
    moment
  • restructuring the program architecture
  • modeling and documenting the project
  • Making PlebNet communicate back over IRC

Documentation:

Team 6:

Starting November 2018 the Plebnet project is being improved by:

Documentation:

@synctext
Copy link
Member Author

synctext commented Dec 2, 2018 •
edited
Loading

Impressive overview! 25 developers...

@synctext
Copy link
Member Author

synctext commented Dec 3, 2018

Meeting minutes. Current status: simple learning and reproduction operational. Created wealth of 20GBytes in 2 days and creating market orderbook entries.

Continuous procurement bot buys from the self-replicating bots. This is still under construction.
Next problem "derive efficient representations of the environment" for Q-tables. When each step in learning takes 1 month, you learn very slowly. Possible solution: robot learning gossip for parallel learning.

Upcoming sprint: agents will have a character class which determines: appetite for risk, long-term versus short term thinking, stubbornness versus flexible. Q-tables in general and CloudOmate in general. CI?

@Jaapp- Jaapp- mentioned this issue Dec 3, 2018
Merged
@synctext
Copy link
Member Author

synctext commented Dec 10, 2018 •
edited
Loading

Meeting minutes: Current status: initial Q-table code created, multiple market strategies, and partial fulfillment of orderbook entries.

Upcoming sprint goal: end-to-end test

Idea of collective robot learning seems viable. First mechanism is message exchange between robots, assuming full trust, complete honesty and no attackers. Use IPv8 tutorial to publish your selected public-key, announce using our DHT, and receive incoming messages. Remove IRC server based mechanism in Plebnet. Use messages in Protocol buffers serialisation. Example demo messages to implement: "full Q-table sharing", keep it simple and atomic. Only share after you learned something and "edited it".

To call it "deep reinforcement learning" we need a neural network and historical performance data, yield of market strategies, and hardware performance (intro). Future sprint could use a testnet and orderbook replay, see our prior work with limit orders. Do the bots need to crawl the IPv8 market community to build transaction history database of last 30days? Please avoid tensorflow or other gigantic libraries, keep it simple and maintainable for easy bot-install (like here).

@synctext
Copy link
Member Author

Meeting notes:

  • finishing prior sprint stuff; QTable, market strategies, VPS selection
  • operational: "Hello World" messages between bots. Towards collective learning.
  • IRC still there for easy debugging
  • initial code for market transaction history; adding Simple Moving Average
  • Two active exit node agents
  • Focus on "collective reinforcement learning" and actual deployment
  • Continuous procurement bot deployment ("buy bot")
  • Plebnet renaming? ('Dolly Bot'?)

@synctext
Copy link
Member Author

Thoughts; please use as basis for your readme.md:

  • Darwinian reinforcement learning
  • "Internet-deployed Darwinian reinforcement learning based on self-replicating"
  • advancement of our own gossip learning done within TUDelft / Szeged collaborative project in 2011
  • breaks with tradition by not having a single trained model
  • scales because it does not attempt to create a single globally consistent model
  • continuously exchange training data; never merge into a single trained model
  • each node or agents has their own unique machine learning model
  • very uncomfortable: probabilistic learning depending on the individual model (Thnx Jan! for this insight)
  • "Darwinator" = Darwin + terminator = heretic scientist + killer robot :-)

@Harapuia
Copy link

I'm still monitoring this issue by the by, If you have any questions or want to yell at me for what we did to cloudomate feel free to shoot a message.

@synctext
Copy link
Member Author

synctext commented Jan 8, 2019

Meeting minutes:

  • This week wrap up the features and test.
  • Future Sprints: end-to-end replication tests with increasing size (feature freeze/limited extensions only?)
  • Now implemented: deterministic QTable selection. Survival/Darwinian performance is enhanced with non-determinism in VPS self-replication. Fancy simulated annealing optimization or just an exponentially deceasing probability to select the No1. choice; No.2 VPS provider with much less probability, etc.
  • non-blind market strategies ongoing work
  • crashed Tribler wallet, untested code for auto-buy-bot.
  • TUN/TAP and is VPN really enabled?

@synctext
Copy link
Member Author

VPN shopping list: "All providers listed here are outside the US, use encryption, accept Bitcoin, support OpenVPN and have a no logging policy".

@synctext
Copy link
Member Author

synctext commented Jan 14, 2019 •
edited
Loading

Next week: end-to-end replication tests (and screen capture of cmdline demonstration on Youtube)?

@MattSkala
Copy link

@synctext
Copy link
Member Author

Meeting note:

  • Please rename repo to something more marketing compatible.
  • {repeat} "Darwinator" = Darwin + terminator = heretic scientist + killer robot :-)
  • DollyNator from Dolly the clone sheep to Terminator; With LOGO as first picture in readme.md:
    3228843714_3dc4859430_b
  • Do an official 0.1.0 release on Github?

@synctext
Copy link
Member Author

@mateicristea88 and @JayDew are thinking of becoming "team 7" for their honor track at TUDelft.

@synctext
Copy link
Member Author

synctext commented Jul 4, 2019 •
edited
Loading

@mateicristea88 and @JayDew. Some semi-random suggested reading for broadening your scientific knowledge base (note, you've asked for a challenge..):

(honor project brainstorm: Facebook alternative requires ultra-reliable data storage. A self-replicating agent that uses agent gossip to offer bullet-proof encrypted personal data hosting under control of a swarm of AI agents {learning performance demand versus performance offered)

@JayDew
Copy link

JayDew commented Jul 29, 2019

@mateicristea88 and @JayDew. Some semi-random suggested reading for broadening your scientific knowledge base (note, you've asked for a challenge..):

* 1998: [Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria](http://www.opim.wharton.upenn.edu/~sok/papers/e/erev-roth-AER-1998.pdf)

* [Reinforcement Learning Dynamics in Social Dilemmas](https://pdfs.semanticscholar.org/6517/ac9d6d3da823b258128c520bbf41fed20200.pdf)

* [SybilBelief: A Semi-Supervised Learning Approach for Structure-Based Sybil Detection](https://arxiv.org/abs/1312.5035)

* [IoT security techniques based on machine learning](https://arxiv.org/pdf/1801.06275)

* [The latent community model for detecting sybil attacks in social networks](http://cmj4.web.rice.edu/NSF/paper.pdf)

* [An Adaptive Autonomous Bidding Agent](https://arxiv.org/pdf/1106.0678.pdf)

(honor project brainstorm: Facebook alternative requires ultra-reliable data storage. A self-replicating agent that uses agent gossip to offer bullet-proof encrypted personal data hosting under control of a swarm of AI agents {learning performance demand versus performance offered)

@mateicristea88 and @JayDew.

1998: Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria: Very interesting paper about game theory and equilibrium, also well written, would enjoy more articles like it.
Reinforcement Learning Dynamics in Social Dilemmas: Good follow up to Roth and Erev, slightly too teoretic, less applied.
SybilBelief: A Semi-Supervised Learning Approach for Structure-Based Sybil Detection: didn't have time to read
IoT security techniques based on machine learning: Didn't enjoy it, too many references to other papers, not a lot of original ideas, just running some benchmarks and comparing them
The latent community model for detecting sybil attacks in social networks: Also very interesting, had a bit of trouble with the more advanced statistics (multivariate distributions and bayesian inferences). Cool idea, reading it was pretty enjoyable.
An Adaptive Autonomous Bidding Agent: easy to read, cool, could be used pretty directly to create a tribler bidding agent.

We would also want to inform that we are on track with the Honors program, having achieved all 60 ECTs and an average of 8.

@synctext
Copy link
Member Author

Great to hear the bsc study is also going above-nominal. As a next step, please try to find 5 next-step papers in the AI sub-topics that triggered your interest when reading these papers.

@synctext
Copy link
Member Author

Distributed machine learning:

Plus some links that are relevant for a possible distributed-Facebook based on a swarm of AI agents:

@synctext
Copy link
Member Author

synctext commented Sep 2, 2019 •
edited
Loading

reposting the basic trustchain paper: https://pure.tudelft.nl/portal/files/47579639/paper.pdf
1968 classic: http://pages.mtu.edu/~asmayer/rural_sustain/governance/Hardin%201968.pdf
reputation 2002: https://cseweb.ucsd.edu/groups/csag/html/teaching/cse225s04/Reading%20List/E-bay-Empirical-BodegaBay.pdf
2005: http://www.econ.ucla.edu/sboard/teaching/tech/dellarocas_2005.pdf
https://dl.acm.org/citation.cfm?id=3236008

@MattSkala
Copy link

Accountless, API-driven VPS hosting for Bitcoin:

@synctext
Copy link
Member Author

idea. Fundamental AI research as the key focus of you honors program. Gossip Learning as a Decentralized Alternative to Federated Learning from 2013 and also this one from 2019: http://www.inf.u-szeged.hu/~jelasity/cikkek/dais19a.pdf

ToDo: understand, review, implement, and evaluate for usefulness. What is the performance, scalability, robustness to errors, and attacks through false gossip?

@mateicristea88
Copy link

  • papers have been mostly read, had some issues with SVM equations, need to read more about that
  • basic implementation in java, with merging models by averaging and updating with linear gradient descent; able to converge (UM and MU) on linear regression

@synctext
Copy link
Member Author

Great having the SVM basics up and running. Always focus on the end-2-end architecture, not polish individual components. Read about making the possible step to an illustrative use-case:

Concrete next step: 1) publish code 2) convergence diagram, 3) make cooler: real sockets, 100 machines, etc. 49) use DNA dataset

@MattSkala Do you like the idea of a proof-of-principle that you can transfer your DNA (or synthetic generated data) through Bluetooth for an hour (slow), then run SVM on them, meet others phones, and convergence slowly the model? "A social AI network", "Internet-Free AI". (collaborate with @mateicristea88 @JayDew or focus on your own project; also fine)

@synctext
Copy link
Member Author

synctext commented Oct 28, 2019 •
edited
Loading

Welcome new honor students! @dandreescu @ioanasv Please also read the starting material from 4th of July before the next meeting. We then can see if you like this stuff or put you on another direction. Directions: economic-side, trustchain-core, hardware/embedded, streaming, social/trust/reputations, security, social networks, gossip AI, etc.

@synctext
Copy link
Member Author

Suggested reading:

@synctext
Copy link
Member Author

And another member of the lab :-) @MateiAnton Two suggested directions for reading:

@synctext
Copy link
Member Author

@mateicristea88 @JayDew Trustchain by requirement should be usable for your Robot-Gossip project. If it fails, we need to learn from Trustchain1 and design Trustchain2...!

@MattSkala is working on Native Java work getting trustchain up and running. He is working on getting code ready for the Blockchain Engineering master course. #4863 (comment)

As honor students you have not yet worked with Trustchain, please dive into it. So a practical sprint these weeks. If you browsed the Android code and workings a bit, please organise a chat with Matouš to sync with his work and roadmap for coming 2 months.

For now the "Decentral AI" could be a completely isolated button in the app. Similar to how we did "Scan Passport" in the Play store app "Trustchain". For demo purposes, that it simply runs a demo script. Simple thing, like calculating human DNA variation metrics.

Draft API:

  • connected peer list in Trustchain
  • send-robot-gossip-message()
  • receive-robot-gossip-message()

@synctext
Copy link
Member Author

@MateiAnton Please read https://www.ifi.uzh.ch/ce/publications/Accounting_Mechanisms.pdf
and others please: #4039 (comment)
Also robot economy work might be interesting: #4039 (comment)

@synctext
Copy link
Member Author

synctext commented Dec 9, 2019

Update meeting:

  • busy with exams/study, not much time this sprint for project
  • message exchange and first Java/Android screen.
  • got peer_list() operational
    Ideas:
    • input of a few number on the GUI
    • gossip this around and show result in real-time
    • use Trustchain for agent discovery/meeting others

@mateicristea88
Copy link

mateicristea88 commented Feb 26, 2020 •
edited
Loading

Update:

  • very basic way to input numbers on gui
  • display nice graph of accuracy of machine learning
  • troubles with Trustchain agent discovery, keeping it local for now

link to repository:
https://github.com/mateicristea88/trustchain-android

@MattSkala
Copy link

@mateicristea88 In case you are trying to use trustchain-android for communication, please switch to kotlin-ipv8 which should be much more reliable and easier to use.

@synctext synctext mentioned this issue Mar 2, 2020
Closed
@mateicristea88
Copy link

Update:

  • not a lot, busy with exams and studying from home
  • getting comfortable with kotlin

@synctext
Copy link
Member Author

Sprint update in the age of Corona.

  • trying to spend 1 day per week
  • first priority is getting code into https://github.com/Tribler/trustchain-superapp
  • cleanup and making available as a general library
  • Monday 20 April at least 1 team will starting expanding this for movie/Youtube-like recommendation.
  • critical they having documented and stable code by week 3: Monday 4th of May 2020

@mateicristea88
Copy link

Update:

-got code into superchain-kotlin-app

-not very clean however, we still have some java(not kotlin) code
-also, for some values it deosn't converge
-pushed to local fork: https://github.com/mateicristea88/trustchain-superapp

@synctext
Copy link
Member Author

Sprint progress:

  • great progress, still local ML, prepared for full distributed AI kernel: zero-server collective learning
  • please do a Pull Request into the superapp (dao branch)
  • do first PR before 30 April, we're doing bit of press exposure. @MattSkala
  • rename from "DNAjava" to "distributed AI kernel"
  • march 2021 honor project presentation

@synctext
Copy link
Member Author

This is now officially a hot topic!
Best overview paper I've seen: Adversary-resilient Distributed and Decentralized Statistical Inference and Machine Learning. It is hot because of the entire special issue publication with lots of similar articles. That great overview paper includes Recent Advances Under the Byzantine Threat Model:
image

@mateicristea88 Please do a PR of your distributed AI kernel to the master branch of the superapp. We're doing a release this week and have journalist to talk about it.

@synctext
Copy link
Member Author

synctext commented Jun 18, 2020 •
edited
Loading

Honor project update @mateicristea88 @JayDew. ToDo determine direction for May 2021 wrap-up, required to produce a scientific paper. We now have @awrgold @Tim-W on board. They can build upon your library, deploy it to real users, provide feedback on usefulness, and you can see the live gossip learning network. @awrgold is working toward a DAO-based investment agent with ML and market history datasets.@Tim-W can use your library for music discovery & recommendation.

State of the art: Automating the end-to-end lifecycle of Machine Learning applications. The distributed case remains unsolved. The tough open problem is in security. First deploy, tweak your distributed-ai-kernel to perfection, and then try to attack yourself and think of defences? Fascinating literature on Sybil attack for model poisoning. "Mitigating Sybils in Federated Learning Poisoning". Note that Sybil attacks are real, especially in social media with 4.6 million fake friendship requests. Use a [0,1] trustworthiness estimator of strangers you meet to fix this?

@synctext
Copy link
Member Author

synctext commented Jul 22, 2020 •
edited
Loading

@mateicristea88 @JayDew Superapp is moving forward rapidly, please keep up ;-) Deployed on Google Play store.
May 2021 wrap-up, define concrete goals. Possible phasing

  • July-December: Perfection of distributed AI kernel
  • December-May: dive into the science (define scientific direction at end of 2020)

@synctext synctext closed this as completed Oct 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rwblokzijl rwblokzijl

@MattSkala MattSkala

@Jaapp- Jaapp-

@andrebreis andrebreis

@rpytel1 rpytel1

@HusainKapadia HusainKapadia

Labels
type: MSc course work
Milestone
No milestone
Development

No branches or pull requests