Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meeting Notes / Project Diary #1

Open
landgenoot opened this issue Aug 6, 2019 · 60 comments
Open

Meeting Notes / Project Diary #1

landgenoot opened this issue Aug 6, 2019 · 60 comments

Comments

@landgenoot
Copy link
Member Author

landgenoot commented Aug 6, 2019

Profile will be stored on the client side. This makes it possible to gather more relevant details. How to transfer the profile to the advertiser without revealing too much details.
This makes it possible to deliver ads which are more relevant, so that they will make more revenue.

Ways to

  • Track
  • Modify profile
  • Supply better ads

Exposé,

  • Problem
  • State of the art
  • Explain concept

@landgenoot
Copy link
Member Author

image

@landgenoot
Copy link
Member Author

landgenoot commented Sep 17, 2019

https://www.adex.network
Is not really decentral, but uses the Ethereum network for transparency. Implements microtransactions. Business model not clear?

https://adbank.network/
Features almost everything discussed. Business model seems to be the ICO. Should be live by the end of the year. Total supply of tokens around 5,5 million EUR
The problem with this system is that it is not academic. All software seems to be open source, but the research that is done to come up with this product stays proprietary. This makes it hard to extend or improve the system.

@landgenoot
Copy link
Member Author

image

@landgenoot
Copy link
Member Author

@landgenoot
Copy link
Member Author

image

@landgenoot
Copy link
Member Author

Research proposal as expected.
Graduate seminar at 6.11
Pagerank algorithm for reputation

Presentation:

https://www.snet.tu-berlin.de/fileadmin/fg220/theses/master-snet_169.potx
Payment methods is too much
Evaluation: Simulate real life scenario
Maybe time schedule

@landgenoot
Copy link
Member Author

landgenoot commented Oct 8, 2019

Feedback on slides:

Problem & statement

Define definitions
Explain rolmodel
Explain why current model is not effective
New opportunities crypto
Why unpaid content Kernproblem

Motivation

Explain problem with current systems,

Evaluation

Add performance

Related work

Maybe crypto paper

@landgenoot
Copy link
Member Author

photo_2019-10-22_16-01-01

@landgenoot
Copy link
Member Author

landgenoot commented Nov 6, 2019

Some feedback from Prof. Dr. Küpper:

30 minute agressive monologue about:

  • The presentation is a BWL-presentation, it is only about business models.
  • Only one slide about architecture.
  • It is only about what is it not.
  • It is not about privacy? It is not about micropayments?
  • The presentation raises more questions than that it answers.
  • I do not understand it.
  • If it is about micropayments, I should list paypal and why it is bad.
  • Why is a URL redirect bad?

Current considerations:

  • Drop the idea and ask Philip for an assignment
  • Add the crypto part, so that it is an entire solution, which is understandable can be provided. Will make the project hudge.

Questions to Philip:

  • Please be more critical in the future
  • Can I discuss the initial presentation with Prof Dr. Küpper on beforehand to avoid any further disgrace?
  • "Implementation of a Voice User Interface for a Crowdsourcing Service " Why is this academic at all? Connecting three API's? Why doesn't he get any comments at all?

@landgenoot
Copy link
Member Author

@landgenoot
Copy link
Member Author

landgenoot commented Nov 14, 2019

20191114_135759

@landgenoot
Copy link
Member Author

landgenoot commented Nov 28, 2019

  • Move Targed advertising diagram to problem

  • Merge micropayments and subscriptions

  • Requirements after desired solution

  • Move I am paying to publisher

  • Brave browser add logo

  • challenges ad ...

  • Add blockchain technology

  • Or add extra slide to do desired solution

  • How to manage user preferences

  • Strategies

  • Remove concept & design

  • Extend desired solution instead

  • User has a browser

  • Ad brocker is a library

  • Settings page

  • Strategies

  • Desired solution

  • Add steps

  • Add speakers notes

@landgenoot
Copy link
Member Author

photo_2019-12-02_14-47-33

@landgenoot
Copy link
Member Author

Discuss with Philip the possiblities of the Lightning protocoll.
Some tests with it turned out to be really successful.

The problem is that there is no web based implementation of the lightning protocoll (yet). So for now we will use the node-launcher project which will run a lightning node on the computer of the user. This client is accessible via a REST API and features a lightning wallet.

In this new approach, we will stick to the existing infrastructure, but our javascript library will talk to the node-launcher instead.

This makes it possible to build a fully working prototype, however, we need to drop the zeroconf idea and turn it into a 0.5 conf prototype.

@landgenoot
Copy link
Member Author

Did some further research on the Lightning protocol.
Problem with the protocol is that it is not possible to send any amount to a wallet directly: an invoice should be issued first. This will make the process a lot more complicated, and results in a couple of extra request.

Current decision. Generate the invoice on the fly using the macaron of the publisher.
https://www.npmjs.com/package/@lntools/invoice

@landgenoot
Copy link
Member Author

lightningnetwork/lnd#3795

Invoiceless payments.

@landgenoot
Copy link
Member Author

Coming feature to LND

Accounting-based macaroons, which can make an instance of lnd act almost like a bank for apps: for example, an app that pays to consume APIs whose budget is limited to the money it receives by providing an API/service

@landgenoot
Copy link
Member Author

landgenoot commented Jan 23, 2020

Discussion points Philip:

  • Lightning basic principles / issues
  • No testnet
  • Invoices + solution
  • Virtual environments / machines
  • Trust issue + solution with macaroons
  • Plugin / website?

@landgenoot
Copy link
Member Author

landgenoot commented Jan 25, 2020

The fact that this prototype will work with the bitcoin mainnet is a major advantage.
Will setup multiple VirtualBoxes to demonstrate the system.

Plugin or website -> none of them. Decided that we will built upon the node-launcher system and just expose an arbitrary http port in order to communicate with the publisher.

image

@landgenoot landgenoot changed the title Meeting Notes Meeting Notes / Project Diary Jan 25, 2020
@landgenoot
Copy link
Member Author

landgenoot commented Jan 25, 2020

image

Google Scholar just scrapes PDF's on github for some reason.

image
Not sure why Google thinks it is a PhD-thesis.

@landgenoot
Copy link
Member Author

landgenoot commented Jan 28, 2020

First payment with --keysend succesful.
Next step will be the implementation of the webservice
ezgif-3-d6d3665af2a7

./lncli sendpayment --keysend --dest=027d2456f6d4aaf27873b68b7717c8137aaa8043d687a2113b916a5016e9a880e9 --amt=10 --final_cltv_delta=40

@landgenoot
Copy link
Member Author

landgenoot commented Feb 6, 2020

Problem I

Discussed the problem with keysend and gRPC. We can use the standard CLI as a fallback.

Possible solution to prevent batch domainname fraud could be the mandatory use of real SSL certificates (non let's encrypt).

@landgenoot
Copy link
Member Author

landgenoot commented Feb 7, 2020

Further research on certificates:
There is a difference between domain-, organization and extended certificates.

Domain is like, Letsencrypt only validates that someone is the owner of the domain name.
Organization Validated SSL Certificates require that the organization is registered in some governmental database, which is perfect for this cause.

Certificate validation can be performed easily by doing a cURL request from Python.

@landgenoot
Copy link
Member Author

landgenoot commented Feb 28, 2020

Problem I with gRPC solved. Turned out that the preimage can be created by generating 32 random bytes and the payment_hash is just a hash of the preimage.

Now a simple flask web server is running with one endpoint that supports requesting 10 satoshis

@landgenoot
Copy link
Member Author

landgenoot commented Feb 29, 2020

Questions to Philip:

  1. What kind of license should I use for the project?
  2. Copyright is TUB, right?

@landgenoot
Copy link
Member Author

landgenoot commented Feb 29, 2020

iightning-sprinkle-server is now working with the adjusted reward system.

Reward system

Set a max_hourly_amount, e.g. 40 satoshis.
The first publisher that requests a payment gets 25% of 40, so 10 satoshis (€ 0.00078591).
30 satoshis are left in the bucket, so the next publisher gets 25% of 30, which is 7 satoshis (€0.00055013).
And so on.

The bucket is filled every minute with the max_hourly_amount/60 till max_hourly_amount is reached.

This system makes it impossible to steal all your coins, but only up to the max_hourly_amount. Another possible security measure might be to limit the payment requests to 1 per hour per domain name. However, it is fairly easy to register a large amount of domain names in order to attack this. Futhermore, the organization SSL certificate requirement can be implemented as discussed here.

Localhost and port number

The webservice in bind to 127.0.0.1 in order to make sure it can only be accessed from the local machine and is bound to port number 283733 which is randomized by me typing blind on the number row.

Next steps

The next step will be to implement the publisher-library, which will be written in vanilla JS, in order to keep it small and it won't affect the page speed that much.

@landgenoot
Copy link
Member Author

landgenoot commented Feb 29, 2020

Problem II

Current problem has to do with mixed content. We are not allowed to request our http service, if the publisher domain is on https.

There might be a couple of possible workarounds:

  1. Let the user thrust our self signed certificate. (not user friendly)
  2. Re-use the certificate from lnd which is already trusted (user should have trusted the lightning cert)
  3. Register a domainname which points to 127.0.0.1 and publish the private key, so that every instance of the service can use the same certificate (really hacky)
  4. Only support http for publishers for now (not really 2020-proof)
  5. Generate a certificate and install it to the users' computer (is this possible without sudo rights?)
  6. Thanks to this article about zoom, who is using a similiar trick. You can still load an image from localhost and mixed content. We can reply a status by using several image dimensions. (hacky, but user friendly)

I have verified the trick from number 6 on github pages, which works, so that we don't have to setup a separate server and everything can stay on github.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 1, 2020

Attack vector I

Right now the system relies on an image that is requested on localhost. However, someone might spam an embedded image of the url all over the internet in order to get paid themselves, not the publisher.

Possible solution A

Require the publisher to register a subdomain containing the pubkey, e.g. spiegel.de has to register: 027d2456f6d4aaf27873b68b7717c8137aaa8043d687a2113b916a5016e9a880e9.spiegel.de as a subdomain, which should be used in order to make the request. By using this hacky workaround, it is not possible to embed an image on a public forum that requests the payment.

Possible solution B

Add a TXT record to the DNS which contains the pubkey. Therefore an empty request is enough and Python can find out the pubkey using a Domain Information Groper. This limits the authorization to request a payment to domain-owners.

A single TXT record has a limit of 255-bytes or characters in a single string

DNS records can easily be fetched using:
http://www.dnspython.org/examples.html

@landgenoot
Copy link
Member Author

Feedback Philip:

  • Create more clear example site
  • Manual payment requests.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 3, 2020

Problem III

Referer is not sent if the publisher's website is accessed over https.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer

Edit: solved:
https://developer.mozilla.org/en-US/docs/Web/API/HTMLImageElement/referrerPolicy

@landgenoot
Copy link
Member Author

Just created the certificate checker. Turns out that the type of certificate is stored in the Certificate Policies extension.

If there is a '2.23.140.1.2.2' or '2.23.140.1.1' property present, we are dealing with an OV or EV certificate.

This is now embedded in the isOrganization('https://www.example.com') function, which also supports SNI.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 5, 2020

Last night I was thinking about the approval of the payment.

Right now: I would go for something like this:

  • Users visits publisher for the first time
  • If we cannot connect to localhost:28373 we show ads anyway
  • If we can connect to localhost:28373, we will show an Iframe where the user can decide if he want to pay to hide ads
  • Publisher will be whitelisted and ads will stay hidden.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 7, 2020

Attack vector II

Clickjacking: Iframes are vulnerable to clickjacking.

Possible solution. Show the message from Python with QT, however this makes the experience less seamless.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 7, 2020

Just requested a real Google Adsense account in order to create a working example.

It seems to be possible to hide adsense in a later stage:
https://stackoverflow.com/questions/22410370/remove-adsense-on-mobile

@landgenoot
Copy link
Member Author

landgenoot commented Mar 8, 2020

Now we are communicating via an iframe, it might also be possible to communicate via messages, which makes the hack from Problem II not needed anymore.

myIframe.contentWindow.postMessage('hello', '*');
window.onmessage = function(e){
    if (e.data == 'hello') {
        alert('It works!');
    }
};

@landgenoot
Copy link
Member Author

landgenoot commented Mar 9, 2020

Setup ads on the example page. However somehow my browser is also blocking adsense with my adblocker disabled.

In the VM it works perfectly. Never been so happy to see an ad.
Fresh VM only some traffic to Github.com, so that must be a lonely nerd looking for a girlfriend... :/

image

Next step is checking if we can disable the ads after pageload.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 9, 2020

$('.adsbygoogle').remove()

Seems to do the trick

Next step is to setup an example publisher website. Considering Wordpress or just some static HTML.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 9, 2020

Stumbled upon a nice template, but it was not licensed. Asked the owner for permission:
leogodoyllg/News-Website-Template#1

Later on, I found the original template, which has a MIT license:
https://github.com/imshashikantdev/WTFNews

Will go with the WTFNews template, thanks @imshashikantdev

@landgenoot
Copy link
Member Author

landgenoot commented Mar 9, 2020

Packed homepage full with ads, thanks @google

https://publisher.landgenoot.com

image

@landgenoot
Copy link
Member Author

landgenoot commented Mar 9, 2020

Problem IV

Chrome does block mixed content, which means we cannot load an iframe with localhost embedded.
Everything works fine in Firefox

Possible solutions:

  1. Don't support chrome for now. Lame
  2. Load a new window (like authorizing a paypal payment)
  3. Use QT for the permission question
  4. Add self signed SSL cert to chrome

Will go for the new window now. Seems to work fine.

function requestPermisison() {
    const y = window.top.outerHeight / 2 + window.top.screenY - ( 500 / 2);
    const x = window.top.outerWidth / 2 + window.top.screenX - ( 400 / 2);
    return window.open('http://localhost:28373/request-permission', 'Lightning-Sprinkle', 'toolbar=no, location=no, directories=no, status=no, menubar=no, scrollbars=no, resizable=no, copyhistory=no, noreferrer=no, width='+400+', height='+500+', top='+y+', left='+x);
}

and

<meta name="referrer" content="origin">

In order to find out the hostname of the publisher.

Next steps:

  • Create check if Lightning Sprinkle is running (if so, we can inform the user about the possibilities)
  • Create request-payment page
  • Realize communication between popup and publisher.

@landgenoot
Copy link
Member Author

https://github.com/lightning-sprinkle/lightning-app

Seems to be much more easy to use, because it does use neutrino as a bitcoin node, so that a real btc node is not needed anymore, if I understand it correctly.

Let's see if we can translate everything that we have in Python to JS and bundle it nice to an appimage and .exe

@landgenoot
Copy link
Member Author

landgenoot commented Mar 10, 2020

Wil check if we can run an express app inside the lightning-app. If yes, will discuss it will @philip-raschke next week.

Then it will just be a matter of translating the flask application to express.

Edit:
Koa runs perfectly inside electron. Now checking if it still works when packed into an appimage
Edit2:
It works perfectly packed. So this gives us a great advantage

@landgenoot
Copy link
Member Author

Right now, I will focus on the the JS lib for the publisher. So that we have a working prototype.
Maybe find some time to work on the koa version the rest of the week.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 10, 2020

The system in action:

ezgif-3-d9f66db9cae9

@landgenoot
Copy link
Member Author

https://gist.github.com/bellbind/f65f78a35bbbd4917a8ae4a6b18c5012

Electron supports apps in tray, so this makes it even better.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 12, 2020

image
Now managed to setup my own electron tray application and translated the cert.py and dns.py libs to js. Everything TDD, which is nice, I think.

The tray application now also spins up an lnd service, but does not interact with it.

@landgenoot
Copy link
Member Author

i still think the electron tray application would be a cool proof of concept. However, is is a lot of work because you need to implement a complete wallet. Even things like setting it up an presenting the recovery passphrase are required to run a lightning node.

One possible solution would be to just fork the lightning-app and bake it all into that app. But that means that you need to run an extra app.

Right now I am exploring the possibilities to kind of link the tray application to the lightning-app so that they can work both simultaneously with the same wallet.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 17, 2020

Something that came to my mind right now:

Why not extend this system to any type of media and also embed it into the piracy "industy"?
E.g. plugin where you can support a movie studio directly while browsing a torrent site?

You could even lauch a decentral Spotify.

Something else: It might also be very interesting how Spotify determines the amount that is being paid to the artist. They follow a similar model where you pay an x amount per month, which is distributed across all artist of the songs that have been played.

@landgenoot
Copy link
Member Author

landgenoot commented Mar 24, 2020

Reading some papers:

Blockchain technology / micropayments

Webtracking

Anonimization

Unpaid content / Paywalls

Online advertising

@landgenoot
Copy link
Member Author

The moment you realize that the paper about paywalls that you are trying to read... is behind a paywall.

@landgenoot
Copy link
Member Author

Reached out to the thesis director of this thesis and asked for the PDF:
https://repository.asu.edu/items/48638

Seems to be the only scientific paper about the Basic Attention Token that is written by a third party.

@landgenoot
Copy link
Member Author

Feedback related work:

Rule of thumb 50 papers
More:
Monetization of user generated content

Focus on the other chapters first.

Timeframe: one chapter per week

Performance: Quality of experience usability
Does not distort internet experience.

2 systems: node launcher und other client
Describe alternatives
Zeroconf achievement

@landgenoot
Copy link
Member Author

Decentralized revenue sharing
Distributed revenue sharing

@landgenoot
Copy link
Member Author

landgenoot commented Jun 22, 2020

Untitled Diagram(4)

New diagram

@landgenoot
Copy link
Member Author

landgenoot commented Jun 30, 2020

What's next:

  • Corrections
  • Revenue distribution model
  • Performance testing
  • Abstract
  • Acknowledgement
  • Appendix Exposé
  • Citing more papers

@landgenoot
Copy link
Member Author

landgenoot commented Aug 17, 2020

Thesis defense structure:

  1. Title slide
  2. Problem statement
    a. Unpaid content
    b. Targeted Advertising
  3. Related work
    a. Brave browser
  4. Purpose
  5. Concept & Design / Implementation
    a. Lightning Network
    b. Localhost
    c. SSL Certificates
    d. Dynamic revenue distribution
  6. Short Demo (Video)
  7. Conclusions
  8. Discussion

@landgenoot
Copy link
Member Author

landgenoot commented Aug 31, 2020

Feedback Philip

A lot of bullet points, max 3
More images
Problem statement: Big picture
Targeted advertising is omnipresent
Privacy: Ask the questions
Other way around
Privacy preserving targeted advertising
Pay for the content instead alternative
Cut the middle man

Problem statement & motivation
Discuss the attempts

Disruptive technology

Explain big picture in design beginning

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant