Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General refactoring #11

Merged
merged 211 commits into from
Jan 12, 2023
Merged
Changes from 1 commit
Commits
Show all changes
211 commits
Select commit Hold shift + click to select a range
4e3172a
Merge pull request #5 from cortze/dev
cortze Jul 4, 2022
42a3883
Changed CID_gen.go
FotiosBistas Aug 3, 2022
72f9050
A hardcoded config, new config options
FotiosBistas Aug 3, 2022
880f9e8
Added new CID source types
FotiosBistas Aug 3, 2022
8dabd66
Configured what type of source is to be used
FotiosBistas Aug 3, 2022
79197d2
Moved cidsource to cid_source.go
FotiosBistas Aug 3, 2022
165c212
what
FotiosBistas Aug 3, 2022
c78c228
Deleted the struct
FotiosBistas Aug 3, 2022
b47cc99
Naming Convention
FotiosBistas Aug 3, 2022
15699aa
Merge branch 'cortze:master' into CLI-refactoring
FotiosBistas Aug 3, 2022
124ca70
First steps into reading from a file
FotiosBistas Aug 3, 2022
13f2f8b
Implemented the reading from file
FotiosBistas Aug 3, 2022
3692143
Added proper error handling
FotiosBistas Aug 3, 2022
c897b95
Moved file close to appropriate place
FotiosBistas Aug 4, 2022
f47f80d
Added already published option in the config
FotiosBistas Aug 4, 2022
35fbf8f
Changing some variable names and adding comments
FotiosBistas Aug 4, 2022
8ff06eb
added some comments
FotiosBistas Aug 4, 2022
2ea9a68
variable names
FotiosBistas Aug 4, 2022
018e2e6
added some comments
FotiosBistas Aug 4, 2022
ad1194e
comments
FotiosBistas Aug 4, 2022
b8b33e9
comments and variable names
FotiosBistas Aug 4, 2022
4a8b27f
Created a new file containing the CID queue
FotiosBistas Aug 5, 2022
f7b2407
comment
FotiosBistas Aug 5, 2022
427a4d9
Created generate CIDs method
FotiosBistas Aug 5, 2022
957eda9
Created provide and provide_process method
FotiosBistas Aug 5, 2022
feedfad
Described the process taken by the tracker
FotiosBistas Aug 5, 2022
ebf0c95
Added todo for the source
FotiosBistas Aug 5, 2022
7ec79d1
Added Comments
FotiosBistas Aug 5, 2022
78f494d
comments
FotiosBistas Aug 5, 2022
e680727
comments
FotiosBistas Aug 5, 2022
d8e1c7e
comment
FotiosBistas Aug 5, 2022
54e7842
Broke into methods and added comments
FotiosBistas Aug 6, 2022
e90ad7e
added comments
FotiosBistas Aug 7, 2022
9ad24e4
Created Publisher ,Discoverer structs and tracker interface
FotiosBistas Aug 8, 2022
6b37149
Cid Tracker Completion and Notifier in the send request
FotiosBistas Aug 9, 2022
7ee401f
Embedded peer.AddrInfo into PeerInfo
FotiosBistas Aug 9, 2022
29dedfa
For loop for inserting peers
FotiosBistas Aug 9, 2022
f1b419f
run for the discoverer
FotiosBistas Aug 9, 2022
667d723
variable names
FotiosBistas Aug 9, 2022
c496cac
Json types
FotiosBistas Aug 13, 2022
dfdcf70
Implemented reading from json file
FotiosBistas Aug 13, 2022
1aeca8a
Removed Discoverer
FotiosBistas Sep 18, 2022
f637932
Created new file for publisher
FotiosBistas Sep 18, 2022
87ca788
General way of generating cids
FotiosBistas Sep 18, 2022
9f1d138
Update cid_source.go
FotiosBistas Sep 18, 2022
8600d3f
Read CIDs function
FotiosBistas Sep 18, 2022
e714c80
Update
FotiosBistas Sep 18, 2022
850a9f7
Created CID discoverer
FotiosBistas Sep 24, 2022
132b1d6
added a field to the struct
FotiosBistas Sep 24, 2022
67de563
Add address to peerstore
FotiosBistas Sep 26, 2022
8d751bc
Changes for already-published-cids flag
FotiosBistas Oct 2, 2022
d35c258
updated imports and go mod file
FotiosBistas Oct 2, 2022
ea6e5ec
Use the default config struct
FotiosBistas Oct 2, 2022
89e54d2
Json config file
FotiosBistas Oct 2, 2022
8ad2b06
read config from json file
FotiosBistas Oct 2, 2022
17f6c01
fixed bug not recognized cid source
FotiosBistas Oct 2, 2022
441652e
example config json file
FotiosBistas Oct 2, 2022
05b76d8
print debug
FotiosBistas Oct 2, 2022
b887060
bug wrong ctx given as a parameter
FotiosBistas Oct 2, 2022
cd764e6
JsonFileCidSource struct
FotiosBistas Oct 4, 2022
b1d8554
added json file source
FotiosBistas Oct 4, 2022
2e904d8
changes for the GetNewCidReturnType
FotiosBistas Oct 4, 2022
36c21d5
debug messages
FotiosBistas Oct 4, 2022
c7a6b17
discoverer not discover
FotiosBistas Oct 4, 2022
04d6723
removed wrong comments
FotiosBistas Oct 4, 2022
d7e71c0
required field already published cids
FotiosBistas Oct 4, 2022
7931db7
collision with package name and variable name
FotiosBistas Oct 4, 2022
72b7d80
return source types from the enum
FotiosBistas Oct 4, 2022
09f7fe7
fixed json type issues with multiaddresses string
FotiosBistas Oct 4, 2022
474ebd1
changed to errorF
FotiosBistas Oct 4, 2022
5c51fef
changed the way the cid file is read
FotiosBistas Oct 4, 2022
b9c7626
fixed bug where already-published-cids flag was not being set
FotiosBistas Oct 4, 2022
3b98999
moved cid source interface to cid-source directory
FotiosBistas Oct 6, 2022
a54bb21
moved bitswap-cid-source to cid-source directory
FotiosBistas Oct 6, 2022
61b77be
moved json-file-source to cid source directory
FotiosBistas Oct 6, 2022
47d141c
moved to new directory
FotiosBistas Oct 6, 2022
2ff833f
exported open functions
FotiosBistas Oct 6, 2022
f4b14e1
added cid-source package
FotiosBistas Oct 6, 2022
3a2aaff
added support for multiple files
FotiosBistas Oct 6, 2022
803efcd
bug fix null pointer exception in iter
FotiosBistas Oct 7, 2022
5aead3a
added proper wait groups and closing the host of the publisher
FotiosBistas Oct 7, 2022
87afe79
added support for two seperate hosts
FotiosBistas Oct 7, 2022
b34ceb0
temporary error in json naming
FotiosBistas Oct 7, 2022
144770f
removed this is hacky and terrible (it's probably not)
FotiosBistas Oct 7, 2022
8935836
fixed not initializing pinger host for the discoverer
FotiosBistas Oct 7, 2022
011974e
changed run cmd flags
FotiosBistas Oct 7, 2022
c99ea71
removed bloating debug message
FotiosBistas Oct 7, 2022
bb8b3ab
removed bloating debug message
FotiosBistas Oct 7, 2022
2c87bf2
receive only channel
FotiosBistas Oct 7, 2022
f091e7a
receive only channel
FotiosBistas Oct 7, 2022
86d4fe6
examplejsonfiles
FotiosBistas Oct 8, 2022
d7db003
changed json name
FotiosBistas Oct 8, 2022
441fb80
test for file openings
FotiosBistas Oct 8, 2022
c5c8b37
added test for get new cid
FotiosBistas Oct 8, 2022
bee8ebc
added test for get new cid for simple json files
FotiosBistas Oct 8, 2022
ec0246a
debugging
FotiosBistas Oct 8, 2022
f104219
added getnewcid for encoded json files
FotiosBistas Oct 8, 2022
0203145
added test for random cid source generation
FotiosBistas Oct 8, 2022
5343f32
added state variables inside random cid struct in order to stop when …
FotiosBistas Oct 8, 2022
37be290
debug and error messages
FotiosBistas Oct 8, 2022
0ab2977
generalized generateCids method
FotiosBistas Oct 8, 2022
e4ec155
changes due to generalization of generatecids method
FotiosBistas Oct 8, 2022
11c0ca1
added test get new cid with channels
FotiosBistas Oct 9, 2022
8916c91
bug fix where workers were not initialized for textfilesource,jsonfil…
FotiosBistas Oct 9, 2022
1d8f5a6
bug where some pids couldn't be parsed
FotiosBistas Oct 9, 2022
0734411
removed publisher fields from tracker
FotiosBistas Oct 9, 2022
6d7409c
removed publisher fields from tracker
FotiosBistas Oct 9, 2022
3f211ea
proper API for putting provider in peerstore
FotiosBistas Oct 9, 2022
b8dc019
removed TODOs
FotiosBistas Oct 9, 2022
e85e156
removed publisher field from tracker
FotiosBistas Oct 9, 2022
bf70755
corrected some comments
FotiosBistas Oct 11, 2022
6952a0c
database tests
FotiosBistas Oct 11, 2022
e068496
temp idea for the persisting into the database
FotiosBistas Oct 13, 2022
0982d9e
Bug fix
FotiosBistas Oct 13, 2022
e0d8e94
Moved fetch results to bottom
FotiosBistas Oct 14, 2022
ee88bf7
Bug where default request interval was being set all the time
FotiosBistas Oct 14, 2022
8453bbb
added wg and *models.cidInfo parameter
FotiosBistas Oct 14, 2022
c61cb5d
added flag to finish
FotiosBistas Oct 15, 2022
4163554
Added debug messages and closing of pinger host
FotiosBistas Oct 16, 2022
8fab024
Delete examplejsonfiles directory
FotiosBistas Oct 16, 2022
a7e5409
updated files
FotiosBistas Oct 16, 2022
ea2105f
Merge branch 'CLI-refactoring' of https://github.com/FotiosBistas/ipf…
FotiosBistas Oct 16, 2022
08d57ed
added wait group
FotiosBistas Oct 16, 2022
6f4d54c
Bug fix
FotiosBistas Oct 17, 2022
b099e50
Bug fix
FotiosBistas Oct 17, 2022
a77484b
removed flags
FotiosBistas Oct 17, 2022
1dc7dc4
Bug fix methods we not finishing
FotiosBistas Oct 17, 2022
96c17eb
example json files and some fixes
FotiosBistas Nov 3, 2022
9bbb449
output to txt
FotiosBistas Nov 3, 2022
8cacbb1
bug fix: pinger not shutting down
FotiosBistas Nov 5, 2022
28b9bdc
bug fix: pinger not closing
FotiosBistas Nov 5, 2022
21397e2
now it works
FotiosBistas Nov 5, 2022
76b77ef
remove examples of OP providers .jsons
cortze Nov 8, 2022
d6227a9
remove priv-key flag from config
cortze Nov 8, 2022
c6f299d
make naming homogeneus + add method to TrackableCid
cortze Nov 8, 2022
9f18b3a
make naming homogeneus + remove priv-key reading from config
cortze Nov 8, 2022
5dcdfe5
homogeneize src naming + rename to TrackableCid
cortze Nov 9, 2022
c4e4b8a
rename + fix missing 1 CID + keep connections open
cortze Nov 9, 2022
ad93f50
update ctx to publisher.ctx (compilation error)
cortze Nov 9, 2022
9714f77
fixes
FotiosBistas Nov 11, 2022
bd05a9b
New json fields and important log
FotiosBistas Nov 17, 2022
bf40169
added user agent fix in the pr holder
FotiosBistas Nov 17, 2022
efa5956
Agent Version
FotiosBistas Nov 17, 2022
342d73f
newest example json file
FotiosBistas Nov 17, 2022
a910b59
Delete cmd_output.txt
FotiosBistas Nov 18, 2022
bdf45f3
analyzer stuff
FotiosBistas Nov 18, 2022
bf7c469
Delete cmd_output3.txt
FotiosBistas Nov 19, 2022
456db10
Delete cmd_output3.zip
FotiosBistas Nov 19, 2022
d1dd959
corrected ream me
FotiosBistas Nov 21, 2022
12bea17
Merge branch 'CLI-refactoring' of https://github.com/FotiosBistas/ipf…
FotiosBistas Nov 21, 2022
1c6dc3d
removed old analyzer scripts
FotiosBistas Nov 21, 2022
b24dae6
Update README.md
FotiosBistas Nov 21, 2022
9ea37e6
Update README.md
FotiosBistas Nov 21, 2022
d1928cc
add flag for port
cortze Nov 29, 2022
a5b9e39
update user agents + add backoff error to list
cortze Nov 30, 2022
0bc5192
update grace periods
cortze Nov 30, 2022
c9c3d10
bug fix: change name of parser due to mix up with py module
FotiosBistas Dec 4, 2022
15881c6
Delete parser.py
FotiosBistas Dec 4, 2022
e9bc52a
multiple cid generation lines
FotiosBistas Dec 4, 2022
cca0048
added encoding utf 8
FotiosBistas Dec 4, 2022
538a523
bug fix: max len instead of l
FotiosBistas Dec 7, 2022
d85dd2e
custom parser and path to logs
FotiosBistas Dec 7, 2022
f599247
changed the plots
FotiosBistas Dec 7, 2022
7e02316
update dependencies
FotiosBistas Dec 9, 2022
6f1ca42
Update dependencies + update kad-dht + tidy go.mod
cortze Dec 13, 2022
5976d3f
update gitignore and analyzer
cortze Dec 13, 2022
900bbaa
update error parser + prevent dialbackoff
cortze Dec 13, 2022
f4e57c9
update Host params to prevent dial backoffs
cortze Dec 14, 2022
cdf30a8
add port to Hoarder configuration
cortze Dec 14, 2022
962bdb5
Update README.md
FotiosBistas Dec 14, 2022
0f17f9f
change make file
FotiosBistas Dec 14, 2022
c3438e5
Add ForceDirectDial to the host context to prevent backoffs
cortze Dec 14, 2022
2520b8a
add error distribution graph
cortze Dec 15, 2022
550ecfa
Delete .idea directory
FotiosBistas Dec 15, 2022
8581932
Update .gitignore
FotiosBistas Dec 15, 2022
c47778b
http server implementation
FotiosBistas Jan 6, 2023
7aac52b
http server get new cid
FotiosBistas Jan 7, 2023
d17dfb6
added dummy trackable cids
FotiosBistas Jan 7, 2023
bd8c02a
block min time and log messages
FotiosBistas Jan 7, 2023
64d1969
http listener
FotiosBistas Jan 7, 2023
ce6618b
waiting time and add cid to db
FotiosBistas Jan 7, 2023
98ff19a
generate cids http
FotiosBistas Jan 8, 2023
74c7ed9
http source with provider records
FotiosBistas Jan 8, 2023
bd66efe
removed lock redundant?
FotiosBistas Jan 8, 2023
8d1b8d4
testing
FotiosBistas Jan 8, 2023
fc25679
updated testing
FotiosBistas Jan 8, 2023
7dc6172
created queue
FotiosBistas Jan 8, 2023
afcdfef
updated testing
FotiosBistas Jan 8, 2023
f79bfc9
Update http_server_test.go
FotiosBistas Jan 8, 2023
ada0020
changed error position
FotiosBistas Jan 8, 2023
bad07e7
removed not in time to ping
FotiosBistas Jan 8, 2023
7f19e4c
format cmd
cortze Jan 4, 2023
ed426cc
update models to make study based on time - not in pings
cortze Jan 9, 2023
5526927
fix retrievability bug
cortze Jan 9, 2023
bf22980
first libp2p host config + study based on time - not in pings
cortze Jan 9, 2023
81edbe4
update dependencies
cortze Jan 9, 2023
01788da
set dial timeout + add new registered errors
cortze Jan 9, 2023
14dc1ec
update analyzer scripts
cortze Jan 9, 2023
c2897d3
Create log1.txt
FotiosBistas Jan 9, 2023
3732357
Delete log1.txt
FotiosBistas Jan 9, 2023
34c4480
Merge remote-tracking branch 'origin/feat/fix-libp2p-configuration' i…
FotiosBistas Jan 9, 2023
d1831e8
synced up http server to latest config
FotiosBistas Jan 9, 2023
5c0f9df
fixed bug in test
FotiosBistas Jan 9, 2023
63c15f6
update go mod
FotiosBistas Jan 9, 2023
637ed8a
added publication time to trackable cid
FotiosBistas Jan 9, 2023
fe9ea1b
graceful shutdown of http server
FotiosBistas Jan 9, 2023
727f98b
publication time addition
FotiosBistas Jan 10, 2023
92a89ed
bug fix: publication time was not getting parsed, added time standard
FotiosBistas Jan 10, 2023
24baa49
bug fix: http server proper shutdown
FotiosBistas Jan 10, 2023
1633d58
added path to http connection
FotiosBistas Jan 12, 2023
5b8a264
Update http_server.go
FotiosBistas Jan 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
added test for get new cid
FotiosBistas committed Oct 8, 2022
commit c5c8b37c13a28f9a5fb25bec2282fac6ed0634c1
8 changes: 8 additions & 0 deletions .idea/modules.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 22 additions & 0 deletions pkg/cid-source/json-file_test.go
Original file line number Diff line number Diff line change
@@ -2,6 +2,7 @@ package cid_source

import (
"fmt"
"reflect"
"testing"
)

@@ -63,3 +64,24 @@ func TestOpenMultipleSimpleJSONFiles(t *testing.T) {
fmt.Println(rec)
}
}

func TestGetNewCidSimpleJSON(t *testing.T) {
file, err := OpenSimpleJSONFile("C:\\Users\\fotis\\GolandProjects\\ipfs-cid-hoarder\\examplejsonfiles\\providers.json")
if err != nil {
return
}
for true {
tp, err := file.GetNewCid()
if err != nil {
t.Errorf("error %s while getting new cid", err)
return
}
if reflect.DeepEqual(tp, Undef) {
break
}
}
}

func TestGetNewCidEncodedJSON(t *testing.T) {

}