Merge pull request #1406 from jumpingliu/issue-1404

fix: Replace redis with postgres as default db

docs_src/about.md

@@ -168,20 +168,20 @@ SDKs are language specific; meaning an SDK is written to create services in a pa
 ### Sensor Data Collection
 EdgeX’s primary job is to collect data from sensors and devices and make that data available to north side applications and systems.  Data is collected from a sensor by a device service that speaks the protocol of that device.  Example: a Modbus device service would communicate in Modbus to get a pressure reading from a Modbus pump.  The device service translates the sensor data into an EdgeX event object.  The device service can then either:
 
-1. put the event object on a message bus (which may be implemented via Redis Streams or MQTT).  Subscribers to the event message on the message bus can be application services or core data or both (see step 1.1 below).
+1. put the event object on a message bus (which may be implemented via MQTT).  Subscribers to the event message on the message bus can be application services or core data or both (see step 1.1 below).
 
     ![image](./general/EdgeX_step1-1.png)
 
 2. send the event object to the core data service via REST communications (see step 1.2).
 
     ![image](./general/EdgeX_step1-2.png)
 
-When core data receives the event (either via message bus or REST), it persists the sensor data in the local edge database.  EdgeX uses Redis as our persistence store.  There is an abstraction in place to allow you to use another database (which has allowed other databases to be used in the past).  Persistence is not required and can be turned off.  Data is persisted in EdgeX at the edge for two basics reasons:
+When core data receives the event (either via message bus or REST), it persists the sensor data in the local edge database.  EdgeX uses PostgreSQL as our persistence store.  There is an abstraction in place to allow you to use another database (which has allowed other databases to be used in the past).  Persistence is not required and can be turned off.  Data is persisted in EdgeX at the edge for two basics reasons:
 
 - Edge nodes are not always connected.  During periods of disconnected operations, the sensor data must be saved so that it can be transmitted northbound when connectivity is restored.  This is referred to as store and forward capability.
 - In some cases, analytics of sensor data needs to look back in history in order to understand the trend and to make the right decision based on that history.  If a sensor reports that it is 72° F right now, you might want to know what the temperature was ten minutes ago before you make a decision to adjust a heating or cooling system.  If the temperature was 85° F, you may decide that adjustments to lower the room temperature you made ten minutes ago were sufficient to cool the room.  It is the context of historical data that are important to local analytic decisions.
 
-When core data receives event objects from the device service via REST, it will put sensor data events on a message topic destined for application services.  Redis Pub/Sub is used as the messaging infrastructure by default (step 2).  MQTT or NATS (opt-in during build) can also be used as the messaging infrastructure between core data and the application services.
+When core data receives event objects from the device service via REST, it will put sensor data events on a message topic destined for application services. MQTT is used as the messaging infrastructure by default (step 2). NATS (opt-in during build) can also be used as the messaging infrastructure between core data and the application services.
 
 ![image](./general/EdgeX_step2.png)
 

docs_src/design/ucr/Common Configuration.md

@@ -65,12 +65,12 @@ In the Levski release the additional common security metrics require all service
     #    Gateway="my-iot-gateway" # Tag must be added here or via Consul Env Override can only chnage existing value, not added new ones.
     ...
     [MessageQueue]
-    Protocol = "redis"
+    Protocol = "mqtt"
     Host = "localhost" <override in compose file same for every service>
     Port = 6379
-    Type = "redis"
-    AuthMode = "usernamepassword"  # required for redis messagebus (secure or insecure).
-    SecretName = "redisdb"
+    Type = "mqtt"
+    AuthMode = "usernamepassword"  # required for mqtt messagebus (secure or insecure).
+    SecretName = "mqtt-bus"
     PublishTopicPrefix = <Service Specific>
     SubscribeEnabled = <Service Specific>
     SubscribeTopic = <Service Specific>

docs_src/general/ContainerNames.md

@@ -40,7 +40,7 @@ The following table provides the list of the default EdgeX Docker image names to
 === "Security"
     |Docker image name|Docker container name |Docker network hostname|Docker Compose service name|
     |---|---|---|---|
-    |vault|edgex-vault|edgex-vault|vault|
+    |openbao|edgex-secret-store|edgex-secret-store|secret-store|
     |nginx|edgex-nginx|edgex-nginx|nginx|
     |edgexfoundry/security-proxy-auth|edgex-proxy-auth|edgex-proxy-auth|security-proxy-auth|
     |edgexfoundry/security-proxy-setup|edgex-security-proxy-setup|edgex-security-proxy-setup|security-proxy-setup|
@@ -49,5 +49,4 @@ The following table provides the list of the default EdgeX Docker image names to
 === "Miscellaneous"
     |Docker image name|Docker container name |Docker network hostname|Docker Compose service name|
     |---|---|---|---|
-    |consul|edgex-core-consul|edgex-core-consul|consul|
-    |redis|edgex-redis|edgex-redis|database|
+    |postgres|edgex-postgres|edgex-postgres|database|

docs_src/general/EdgeX_CN.md

@@ -97,7 +97,7 @@ EdgeX has not adopted the [Agile Manifesto](https://agilemanifesto.org/), but th
 
 **Distributed**
 
-EdgeX is a micro service architecture.  Services communicate with each other via REST or message bus and that communication can occur across nodes (aka machines, hosts, etc.).  Services have even been built to wait and continue to attempt to communicate with a dependent service - allowing for some resiliency.  As such, EdgeX is, at its core, distributable. It was designed such that the services could operate largely independently and on top of whatever limited resources are available at the edge.  As an example deployment, an EdgeX device service could run on a Raspberry Pi or smaller compute platform that is directly connected by GPIO to a physical sensor, while the core services are run on an edge gateway, and the application and analytic services (rules engine) run on an edge service.  This would allow each service to maximize the available resources available to the solution.  Having said that, there are some complexities around real world distributed solutions that adopters would still need to solve depending on their use case and environment.  For example, while services can communicate across a distributed set of nodes, the communications between EdgeX services are not secure by default (as would be provided via something like a cloud native service mesh).  Adopters would need to provide for their own means to secure all traffic between services in most production environments.  Service discovery is not fully implemented.  EdgeX services do register with a service registry (Consul) but the services do not use that registry to locate other services.  If a service changed location, other services would need to have their configuration changed in order to know and use the service at its new location.  Finally, latency is a real concern in edge systems.  In addition to service to service communications, most services use stores of information (Redis for data, Vault for secrets, Consul for configuration) which could also be distributed.  These are referred to as backing services in cloud native terminology.  Even if the communications were secure, if these stores or other services are all distributed, then the additional latency to constantly communicate with services and stores may not be conducive to the edge use case it supports.  Each "hop" on a network of distributed services costs and that cost adds up when building solutions that operate and manage physical edge capability.
+EdgeX is a micro service architecture.  Services communicate with each other via REST or message bus and that communication can occur across nodes (aka machines, hosts, etc.).  Services have even been built to wait and continue to attempt to communicate with a dependent service - allowing for some resiliency.  As such, EdgeX is, at its core, distributable. It was designed such that the services could operate largely independently and on top of whatever limited resources are available at the edge.  As an example deployment, an EdgeX device service could run on a Raspberry Pi or smaller compute platform that is directly connected by GPIO to a physical sensor, while the core services are run on an edge gateway, and the application and analytic services (rules engine) run on an edge service.  This would allow each service to maximize the available resources available to the solution.  Having said that, there are some complexities around real world distributed solutions that adopters would still need to solve depending on their use case and environment.  For example, while services can communicate across a distributed set of nodes, the communications between EdgeX services are not secure by default (as would be provided via something like a cloud native service mesh).  Adopters would need to provide for their own means to secure all traffic between services in most production environments.  Service discovery is not fully implemented.  EdgeX services do register with a service registry (Core-Keeper) but the services do not use that registry to locate other services.  If a service changed location, other services would need to have their configuration changed in order to know and use the service at its new location.  Finally, latency is a real concern in edge systems.  In addition to service to service communications, most services use stores of information (PostgreSQL for data, OpenBao for secrets, Core-Keeper for configuration) which could also be distributed.  These are referred to as backing services in cloud native terminology.  Even if the communications were secure, if these stores or other services are all distributed, then the additional latency to constantly communicate with services and stores may not be conducive to the edge use case it supports.  Each "hop" on a network of distributed services costs and that cost adds up when building solutions that operate and manage physical edge capability.
 
 ### Cloud Native Ingredients Not In EdgeX (and why)
 

docs_src/general/PlatformRequirements.md

@@ -4,11 +4,12 @@ EdgeX Foundry is an operating system (OS)-agnostic and hardware (HW)-agnostic Io
 
 === "Memory"
     Memory: minimum of 1 GB
-    When considering memory for your EdgeX platform consider your use of database - Redis is the current default.  Redis is an open source (BSD licensed), in-memory data structure store, used as a database and message broker in EdgeX.  Redis is durable and uses persistence only for recovering state; the only data Redis operates on is in-memory.  Redis uses a number of techniques to optimize memory utilization. Antirez and Redis Labs have written a number of articles on the underlying details (see list below).  Those strategies has continued to [evolve](http://antirez.com/news/128). When thinking about your system architecture, consider how long data will be living at the edge and consuming memory (physical or physical + virtual).
+    When considering memory for your EdgeX platform, take into account your database choice - PostgreSQL is the default in EdgeX 4.0. PostgreSQL is a powerful, open-source relational database management system that offers robust data storage, query capabilities, and performance optimization. PostgreSQL stores data on disk and leverages advanced caching strategies to enhance performance.
 
-    - [Antirez](http://antirez.com/news/92)
-    - [Redis RAM Ramifications](https://redislabs.com/blog/redis-ram-ramifications-part-i/)
-    - [Redis IO Memory Optimization](https://redis.io/topics/memory-optimization)
+    For more information about PostgreSQL's memory utilization and optimization techniques, refer to the following resources:
+
+    - [PostgreSQL Documentation: Resource Consumption](https://www.postgresql.org/docs/current/runtime-config-resource.html)
+    - [Tuning PostgreSQL Memory Parameters](https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server)
 
 === "Storage"
     Hard drive space: minimum of 3 GB of space to run the EdgeX Foundry containers, but you may want more depending on how long sensor and device data is to be retained.  Approximately 32GB of storage is minimally recommended to start.

docs_src/general/ServicePorts.md

@@ -7,7 +7,7 @@ The following tables (organized by type of service) capture the default service
     |core-data|	59880|
     |core-metadata	|59881|
     |core-command	|59882|
-    |redis|6379|
+    |postgres|5432|
     |consul|8500|
 === "Supporting"
     |Services Name|	Port Definition|

docs_src/getting-started/Ch-GettingStartedDockerUsers.md

@@ -48,10 +48,10 @@ Find the Docker Compose file that matches:
 
 |filename|Docker Compose contents|
 |---|---|
-|docker-compose-arm64.yml|Specifies x86 containers, uses Redis database for persistence, and includes security services|
-|docker-compose-no-secty-arm64.yml|Specifies ARM 64 containers, uses Redis database for persistence, but does not include security services|
-|docker-compose-no-secty.yml|Specifies x86 containers, uses Redis database for persistence, but does not include security services|
-|docker-compose.yml|Specifies x86 containers, uses Redis database for persistence, and includes security services|
+|docker-compose-arm64.yml|Specifies x86 containers, uses PostgreSQL database for persistence, and includes security services|
+|docker-compose-no-secty-arm64.yml|Specifies ARM 64 containers, uses PostgreSQL database for persistence, but does not include security services|
+|docker-compose-no-secty.yml|Specifies x86 containers, uses PostgreSQL database for persistence, but does not include security services|
+|docker-compose.yml|Specifies x86 containers, uses PostgreSQL database for persistence, and includes security services|
 |docker-compose-no-secty-with-ui-arm64.|Same as docker-compose-no-secty-arm64.yml but also includes EdgeX user interface|
 |docker-compose-no-secty-with-ui.yml|Same as docker-compose-no-secty.yml but also includes EdgeX user interface|
 |docker-compose-portainer.yml|Specifies the Portainer user interface extension (to be used with the x86 or ARM EdgeX platform)|

docs_src/getting-started/Ch-GettingStartedGoDevelopers.md

@@ -108,8 +108,7 @@ make build
 Several of the EdgeX Foundry micro services use a database.
 This includes core-data, core-metadata, support-scheduler, among others. Therefore, when
 working with EdgeX Foundry its a good idea to have the database up and
-running as a general rule. See the [Redis Quick Start Guide](https://redis.io/topics/quickstart)
-for how to run Redis in a Linux environment (or find similar documentation for other environments).
+running as a general rule. See the [PostgreSQL Quick Start Guide](https://www.postgresql.org/docs/current/tutorial-start.html)for how to run PostgreSQL in a Linux environment (or find similar documentation for other environments).
 
 ### Run EdgeX Services
 

docs_src/microservices/application/details/Triggers.md

@@ -19,7 +19,7 @@ There are 4 types of `Triggers` supported in the App Functions SDK which are dis
 
 An EdgeX MessageBus trigger will execute the pipeline every time data is received from the configured Edgex MessageBus `SubscribeTopics`.  The EdgeX MessageBus is the central message bus internal to EdgeX and has a specific message envelope that wraps all data published to this message bus.
 
-There currently are four implementations of the EdgeX MessageBus available to be used. Two of these are available out of the box: `Redis Pub/Sub`(default) and `MQTT`. Additionally NATS (both core and JetStream) options can be made available with the build flag mentioned above.  The implementation type is selected via the `[Trigger.EdgexMessageBus]` configuration described below.
+There currently are four implementations of the EdgeX MessageBus available to be used. Two of these are available out of the box: `MQTT`(default) . Additionally NATS (both core and JetStream) options can be made available with the build flag mentioned above.  The implementation type is selected via the `[Trigger.EdgexMessageBus]` configuration described below.
 
 ### Type Configuration
 

docs_src/microservices/application/sdk/details/StoreAndForward.md

@@ -30,9 +30,9 @@ Database configuration section describes which database type to use and the info
 !!! example "Example - Database configuration"
     ```yaml
     Database:
-      Type: "redisdb"
+      Type: "postgres"
       Host: "localhost"
-      Port: 6379
+      Port: 5432
       Timeout: "5s"
     ```
 
@@ -66,7 +66,7 @@ One of three outcomes can occur after the export retried has completed.
     Changing Writable.Pipeline.ExecutionOrder will invalidate all currently stored data and result in it all being removed from the database on the next retry. This is because the position of the *export* function can no longer be guaranteed and no way to ensure it is properly executed on the retry.
 
 ## Custom Storage
-The default backing store is redis.  Custom implementations of the `StoreClient` interface can be provided if redis does not meet your requirements.
+The default backing store is PostgreSQL.  Custom implementations of the `StoreClient` interface can be provided if PostgreSQL does not meet your requirements.
 
 ```go
 type StoreClient interface {

docs_src/microservices/application/services/AppServiceConfigurable/details/Profiles.md

@@ -24,7 +24,7 @@ The following profiles and their purposes are provided with App Service Configur
 
 ### rules-engine
 
-Profile used to push Event messages to the Rules Engine via the **Redis Pub/Sub** Message Bus. This is used in the default docker compose files for the `app-rules-engine` service
+Profile used to push Event messages to the Rules Engine via the **MQTT** Message Bus. This is used in the default docker compose files for the `app-rules-engine` service
 
 One can optionally add Filter function via environment overrides