CyberDB is a main memory database of Python. You can use Dictionaries, Lists as data storage and it supports data persistence. Inter-process communication through Socket TCP has extremely high performance. In addition, you can customize your own data structure based on this module to support Gunicorn inter-process communication, distributed computing, model deployment of machine learning, etc.
1.Enter the command window, create a virtual environment, and enter the following commands in turn
Linux and macOS:
python3 -m venv venv #create virtual environment
. venv/bin/activate #Activate the virtual environment
Windows:
python -m venv venv #create virtual environment
venv\Scripts\activate #Activate the virtual environment
- Install CyberDB, enter in turn
pip install --upgrade pip
pip install cyberdb
If your server and client are running in two different project directories, please install CyberDB in the virtual environment of the server and client respectively.
In this module, we can use CyberDict and CyberList instead of dict and list (a TCP-based Dictionaries-like, Lists-like object).
Please enter the root directory of your project
For the first run, please create a separate py file to create a database table when the server is initialized, and save the table information to the local after running the server. (This file is used when creating a table for the first time, and will not run in the future)
from cyberdb import DBServer
server = DBServer()
server.create_cyberdict('dict1') # Create a database table named dict1
server.create_cyberdict('dict2') # Create a database table named dict2
server.create_cyberlist('list1') # Create a database table named list1
server.start(password='123123') # run the server in the background
'''
This run only saves the table during initialization, the password can be set at will, and will not be used later
'''
server.save_db() # to save server data to local
server.stop() # to stop running the server
CyberDB is starting...
Server stopped.
After the server runs, the cyberdb_file directory will be created in the project root directory (do not delete it), and the client configuration file cyberdb_file/config.cdb will be generated (client operation depends on this file). The default file for database persistence is cyberdb_file/backup /data.cdb
Deploying the server
from cyberdb import DBServer
server = DBServer()
server.load_db() # will automatically load 'cyberdb_file/backup/data.cdb'
server.start(host='127.0.0.1', password='123123', port=9980)
'''
Start the server, fill in the running address, password and port number
If server.start(password='123123') is used, it will run at default address '127.0.0.1' and port'9980'
'''
server.set_backup(period=900)
'''
Set database backup, the default time is once every 900s, and the backup will be permanent until the server stops running
If this interface is not called, the data will not be persisted
If you want to stop a running backup, call 'server.set_backup(period=None)'
'''
# If your program will not run permanently in the background, please add the following command to make the program run permanently
import time
while True:
time.sleep(1000000000)
File cyberdb_file/backup/data.cdb loaded successfully.
CyberDB is starting...
The backup cycle: 900s
The default file for database backup is cyberdb_file/backup/data.cdb (this file will be automatically backed up or updated at the set backup time)
Please enter the root directory of your project
Copying the cyberdb_file of the server to the project root directory of the client (if the same project directory is used as the server and the client, there is no need to copy)
Connect to the database
from cyberdb import DBClient
client = DBClient()
client.load_config('cyberdb_file/config.cdb')
'''
Load the configuration file 'config.cdb'
The default value of the configuration file path is 'cyberdb_file/config.cdb' . If the path here is the same, you do not need to fill in the parameters.
'''
db = client.connect(host='127.0.0.1', password='123123', port=9980)
'''
Enter the address, password and port number configured on the server to obtain the connected database instance db
If you use db = client.connect(password='123123'), it will connect to the default address 127.0.0.1
and port 9980
'''
Manipulate CyberDict and CyberList (code shown later using Jupyter Notebook)
dict1 = db.CyberDict.dict1
dict2 = db.CyberDict.dict2
list1 = db.CyberList.list1
This command can get the database table created by the server in the format of database's instance.datatype.tablename You can also use the following command to get
dict1 = db['CyberDict']['dict1']
dict2 = db['CyberDict']['dict2']
list1 = db['CyberList']['list1']
The dict1, dict2, and list1 obtained here are all network objects. Calling the object method is to interact with the database remotely.
Add key-value pairs in dict1 and dict2
dict1.put(0, 'dict1')
dict1.put(10, 'test')
dict2.put(0, 'dict2')
To get the corresponding value
dict1.get(0)
'dict1'
dict2.get(0)
'dict2'
We use the ‘show’ method to see the table contents of dict1 and dict2
dict1.show()
{0: 'dict1', 10: 'test'}
dict2.show()
{0: 'dict2'}
The ‘show’ method here can directly extract the data in the table (dict here), and can perform any dict operation. Here the execution is: Copying the variate from the database to the client. If your server and client use the same machine, it will occupy twice the memory. It is not recommended to use the ‘show’ method frequently on the same host.
CyberDict and CyberList support all public methods of dict and list (such as dict.get(key), dict.items(), list.append(v), list.pop(), etc.), using any public method is based on TCP data interaction. So there is no support for iteration, no support for private methods (Python magic methods),and no access to data using dict[key] and list[index] . CyberDB gives corresponding alternatives, please read this article carefully. (If you insist on using magic methods, the corresponding API is provided in CyberDB's official documentation)
Note: Use the ’show‘ method to copy the variate in the table to the client's local. The variate obtained by the ‘show’ method are complete dict or list objects, which can perform any operation of Python. like:
dict1.show()[10]
'test'
We can get the length of CyberDict using the 'get_length' method
dict1.get_length()
2
Deleting a key-value pair using the delete method
dict1.delete(10)
'dict1.delete(10)' is equivalent to using 'del dict1[10]' locally
Performing common operations on CyberList below
Increment list1 to a 5 * 5 all-zeros table
for i in range(5):
list1.append([0 for i in range(5)])
Using the 'show' method to get all the data of list1 (the 'show' method here is the same as CyberDict)
list1.show()
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]
Using the 'update' method to modify the 4th value (subscript 3) of list1 to all 1s. ('update' only works on the first dimension of the table)
list1.update(3, [1 for i in range(5)])
list1.show()
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[0, 0, 0, 0, 0]]
Using the 'update_form' method to modify the 5th value (subscript 4, 4) of the 5th row of list1 to 1. ('update_form' is used to modify the two-dimensional form)
list1.update_form(4, 4, 1)
list1.show()
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[0, 0, 0, 0, 1]]
The 'loc' method can locate a one-dimensional table As follows, using the 'loc' method to locate the 4th row of list1
list1.loc(3)
[1, 1, 1, 1, 1]
Use 'loc_form' to locate a two-dimensional table,the 5th row and 5th column of the table are located here
list1.loc_form(4, 4)
1
Slicing with 'slice'
list1.slice(4, 6)
[[0, 0, 0, 0, 1]]
To get the length of list1
list1.get_length()
5
Regarding the iteration of the list, although you can directly use the 'show' method to get the list and iterate, it will take up more memory (if the CyberList is large). CyberDB has built-in generators that can be used to iterate the CyberList as follows
from cyberdb import generate
for row in generate(list1):
print(row)
[0, 0, 0, 0, 0]
[0, 0, 0, 0, 0]
[0, 0, 0, 0, 0]
[1, 1, 1, 1, 1]
[0, 0, 0, 0, 1]
This method is also suitable for iterating over the 'key' of CyberDict
With CyberDB, memory performance can be fully utilized, and different processes (or even different hosts) can communicate through Python's data structures. For tutorials on custom data structures and machine learning deployment, please refer to the official documentation. Thank you for your support!
CyberLight https://www.cyberlight.xyz/