Expose the Progress interface through the HTTP/JSON API #1092

imobachgs · 2024-03-13T16:30:10Z

Trello: https://trello.com/c/2RvcZBFR/3564-5-expose-the-progress-api-over-http

This PR exposes the Progress API for the Manager and Software services through the HTTP/JSON interface. It is implemented as a pair of functions that can be reused in all services that want to implement such an interface.

A `/SERVICE/progress` endpoint

The progress_router function allows to add a /SERVICE/progress route that exposes the current progress:

{
  "current_step": 4,
  "max_steps": 4,
  "current_title": "Calculating the software proposal",
  "finished": false
}

The events stream

The progress_stream builds an events stream that emits a new event when the current_step changes:

{
  "type": "Progress",
  "service": "org.opensuse.Agama.Software1",
  "current_step": 4,
  "max_steps": 4,
  "current_title": "Calculating the software proposal",
  "finished": false
}

Enabling `ServiceStatus` and `Progress` for the Software service

Additionally, the PR enables the ServiceStatus (implemented #1089) and the Progress for the Software service too.

coveralls · 2024-03-13T16:38:36Z

coverage: 74.351% (-0.05%) from 74.398%
when pulling 859617b on http-progress-api
into 3a6ce33 on architecture_2024.

jreidinger · 2024-03-13T19:49:16Z

rust/agama-server/src/software/web.rs

+            "org.opensuse.Agama.Software1",
+            "/org/opensuse/Agama/Software1",
+        )
+        .await,


well, tokio explicitelly mention to not do multiple merge. See documentation at https://dtantsur.github.io/rust-openstack/tokio/stream/trait.StreamExt.html#method.merge
They suggest to use StreamMap for such case https://docs.rs/tokio-stream/latest/tokio_stream/struct.StreamMap.html

Yes, I want to search for an alternative. Thanks for the hint, I skipped that part of the docs 😉. The StreamMap looks good enough. I was also checking the StreamExt of the futures crate.

jreidinger · 2024-03-13T19:51:23Z

rust/agama-server/src/web/common.rs

+
+async fn progress(State(state): State<ProgressState<'_>>) -> Json<Progress> {
+    let proxy = state.proxy;
+    let progress = Progress::from_proxy(&proxy).await.unwrap();


so it should crash when there is issue with dbus? I think that having some error status would be nicer.

Yes, sure, it is just a draft. There are a couple of unwraps here and there.

rust/agama-server/src/web/common.rs

jreidinger · 2024-03-13T19:58:16Z

rust/agama-server/src/web/common.rs

+) -> Result<ProgressProxy<'a>, zbus::Error> {
+    let proxy = ProgressProxy::builder(&dbus)
+        .destination(destination.to_string())?
+        .path(path.to_string())?


I am not sure if there is advantage of using &str when you inside always needs to create new string. If you have String in method call already, then it can be passed or created outside and just passed.

Well, using &str in the boundaries looks more flexible to me. And, after all, from outside you do not need to know what is going to happen inside. But yes, I could reconsider it.

jreidinger · 2024-03-13T19:59:56Z

rust/agama-server/src/web/event.rs

+    ServiceStatusChanged {
+        service: String,
+        status: u32,
+    },


I expect that they usually come together that Busy and ServiceStatus change?

They are different things: one is the list of busy services, which is only kept by the Manager. The ServiceStatusChanged belongs to each separate service. However, I am thinking that perhaps we are not interested in the BusyServicesChanged event from the web UI. I will check.

* It is replaced by the ProgressStream.

rust/agama-server/src/web.rs

jreidinger

LGTM

Adapt the `WithProgress` to the HTTP-based implementation of the D-Bus progress interface (#1092).

After a few months of work, it is time to merge the `architecture_2024` branch into `master`. It is still a work-in-progress, but all the efforts should be go now against that branch. ## Pull requests * #1061 * #1064 * #1073 * #1074 * #1080 * #1089 * #1091 * #1092 * #1094 * #1095 * #1099 * #1100 * #1102 * #1103 * #1112 * #1114 * #1116 * #1117 * #1119 * #1120 * #1123 * #1126 * #1129 * #1130 * #1131 * #1132 * #1133 * #1134 * #1136 * #1139 * #1140 * #1143 * #1146 ## Other commits * 8efa41f * 9e2dec0

Prepare for releasing Agama 8. It includes the following pull requests: * #884 * #886 * #914 * #918 * #956 * #957 * #958 * #959 * #960 * #961 * #962 * #963 * #964 * #965 * #966 * #969 * #970 * #976 * #977 * #978 * #979 * #980 * #981 * #983 * #984 * #985 * #986 * #988 * #991 * #992 * #995 * #996 * #997 * #999 * #1003 * #1004 * #1006 * #1007 * #1008 * #1009 * #1010 * #1011 * #1012 * #1014 * #1015 * #1016 * #1017 * #1020 * #1022 * #1023 * #1024 * #1025 * #1027 * #1028 * #1029 * #1030 * #1031 * #1032 * #1033 * #1034 * #1035 * #1036 * #1038 * #1039 * #1041 * #1042 * #1043 * #1045 * #1046 * #1047 * #1048 * #1052 * #1054 * #1056 * #1057 * #1060 * #1061 * #1062 * #1063 * #1064 * #1066 * #1067 * #1068 * #1069 * #1071 * #1072 * #1073 * #1074 * #1075 * #1079 * #1080 * #1081 * #1082 * #1085 * #1086 * #1087 * #1088 * #1089 * #1090 * #1091 * #1092 * #1093 * #1094 * #1095 * #1096 * #1097 * #1098 * #1099 * #1100 * #1102 * #1103 * #1104 * #1105 * #1106 * #1109 * #1110 * #1111 * #1112 * #1114 * #1116 * #1117 * #1118 * #1119 * #1120 * #1121 * #1122 * #1123 * #1125 * #1126 * #1127 * #1128 * #1129 * #1130 * #1131 * #1132 * #1133 * #1134 * #1135 * #1136 * #1138 * #1139 * #1140 * #1141 * #1142 * #1143 * #1144 * #1145 * #1146 * #1147 * #1148 * #1149 * #1151 * #1152 * #1153 * #1154 * #1155 * #1156 * #1157 * #1158 * #1160 * #1161 * #1162 * #1163 * #1164 * #1165 * #1166 * #1167 * #1168 * #1169 * #1170 * #1171 * #1172 * #1173 * #1174 * #1175 * #1177 * #1178 * #1180 * #1181 * #1182 * #1183 * #1184 * #1185 * #1187 * #1188 * #1189 * #1190 * #1191 * #1192 * #1193 * #1194 * #1195 * #1196 * #1198 * #1199 * #1200 * #1201 * #1203 * #1204 * #1205 * #1206 * #1207 * #1208 * #1209 * #1210 * #1211 * #1212 * #1213 * #1214 * #1215 * #1216 * #1217 * #1219 * #1220 * #1221 * #1222 * #1223 * #1224 * #1225 * #1226 * #1227 * #1229

imobachgs changed the title ~~Http progress api~~ Expose the Progress interface through the HTTP/JSON API Mar 13, 2024

Base automatically changed from http-manager-srv to architecture_2024 March 13, 2024 17:16

imobachgs force-pushed the http-progress-api branch from 3586de9 to 9dbdc28 Compare March 13, 2024 17:16

jreidinger reviewed Mar 13, 2024

View reviewed changes

rust/agama-server/src/web/common.rs Show resolved Hide resolved

jreidinger reviewed Mar 13, 2024

View reviewed changes

rust/agama-server/src/web/common.rs Show resolved Hide resolved

jreidinger reviewed Mar 13, 2024

View reviewed changes

imobachgs force-pushed the http-progress-api branch 2 times, most recently from 9a7046e to e234b29 Compare March 14, 2024 16:53

imobachgs marked this pull request as ready for review March 14, 2024 16:54

imobachgs added 7 commits March 14, 2024 17:56

Use consts in the manager_service function

967aef3

Add support to expose the Progress API

44cf3bf

Expose the Manager Progress over HTTP

6a40c3e

Drop the EventsProgressPresenter

81fe850

* It is replaced by the ProgressStream.

Expose Software ServiceStatus and Progress APIs

dd88614

Avoid chaining StreamExt::merge calls

2cbdd57

Add error handling to the web::common module

859617b

imobachgs force-pushed the http-progress-api branch from e234b29 to 859617b Compare March 14, 2024 17:57

jreidinger reviewed Mar 15, 2024

View reviewed changes

rust/agama-server/src/web.rs Outdated Show resolved Hide resolved

jreidinger approved these changes Mar 15, 2024

View reviewed changes

imobachgs merged commit 64893e6 into architecture_2024 Mar 15, 2024
2 checks passed

imobachgs deleted the http-progress-api branch March 15, 2024 07:03

imobachgs mentioned this pull request Mar 19, 2024

Adapt to the HTTP-based Progress API #1103

Merged

imobachgs added a commit that referenced this pull request Mar 19, 2024

Adapt to the HTTP-based Progress API (#1103)

3f13f6f

Adapt the `WithProgress` to the HTTP-based implementation of the D-Bus progress interface (#1092).

imobachgs mentioned this pull request May 3, 2024

Merge architecture_2024 branch into master #1177

Merged

imobachgs mentioned this pull request May 17, 2024

Update to release 8 #1232

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose the Progress interface through the HTTP/JSON API #1092

Expose the Progress interface through the HTTP/JSON API #1092

imobachgs commented Mar 13, 2024 •

edited

Loading

coveralls commented Mar 13, 2024 •

edited

Loading

jreidinger Mar 13, 2024

imobachgs Mar 13, 2024 •

edited

Loading

jreidinger Mar 13, 2024

imobachgs Mar 13, 2024

jreidinger Mar 13, 2024

imobachgs Mar 13, 2024

jreidinger Mar 13, 2024

imobachgs Mar 13, 2024

jreidinger left a comment

Expose the Progress interface through the HTTP/JSON API #1092

Expose the Progress interface through the HTTP/JSON API #1092

Conversation

imobachgs commented Mar 13, 2024 • edited Loading

A /SERVICE/progress endpoint

The events stream

Enabling ServiceStatus and Progress for the Software service

coveralls commented Mar 13, 2024 • edited Loading

jreidinger Mar 13, 2024

Choose a reason for hiding this comment

imobachgs Mar 13, 2024 • edited Loading

Choose a reason for hiding this comment

jreidinger Mar 13, 2024

Choose a reason for hiding this comment

imobachgs Mar 13, 2024

Choose a reason for hiding this comment

jreidinger Mar 13, 2024

Choose a reason for hiding this comment

imobachgs Mar 13, 2024

Choose a reason for hiding this comment

jreidinger Mar 13, 2024

Choose a reason for hiding this comment

imobachgs Mar 13, 2024

Choose a reason for hiding this comment

jreidinger left a comment

Choose a reason for hiding this comment

imobachgs commented Mar 13, 2024 •

edited

Loading

A `/SERVICE/progress` endpoint

Enabling `ServiceStatus` and `Progress` for the Software service

coveralls commented Mar 13, 2024 •

edited

Loading

imobachgs Mar 13, 2024 •

edited

Loading