Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development to main #52

Merged
merged 79 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
6aa3ac3
finished synchro test frontend
RalkeyOfficial Oct 17, 2024
ec7b964
Added a isTest argument to just test a sync
bbrands02 Oct 24, 2024
1b15fc2
Test sync endpoint added
bbrands02 Oct 24, 2024
8840910
Merge remote-tracking branch 'origin/development' into feature/CONNEC…
bbrands02 Oct 25, 2024
da6d8ff
Synchronization fixes
bbrands02 Oct 25, 2024
1585dbb
Merge branch 'development' into feature/CONNECTOR-72/synchronizations…
bbrands02 Oct 29, 2024
3eb99a1
Fix sync test endpoint
bbrands02 Oct 29, 2024
65dc38d
fix jobs test endpoint
bbrands02 Oct 29, 2024
eb472a2
Merge fix
bbrands02 Oct 29, 2024
913e07d
Another merge fix
bbrands02 Oct 29, 2024
1185d8e
Added pagination functionality
bbrands02 Oct 30, 2024
a9c035e
Merge remote-tracking branch 'origin/feature/CONNECTOR-72/synchroniza…
bbrands02 Oct 31, 2024
33a5204
Lots of fixes
bbrands02 Oct 31, 2024
41013c4
Merge remote-tracking branch 'origin/development' into feature/CONNEC…
bbrands02 Oct 31, 2024
4e34119
the small fixes
rubenvdlinde Nov 3, 2024
0e386ab
Merge remote-tracking branch 'origin/development' into fix/small-fixe…
rubenvdlinde Nov 3, 2024
b62d075
Give a bit of controll on the location of the id and resutls in the s…
rubenvdlinde Nov 3, 2024
25c6dd8
Set the result and id positon from the form
rubenvdlinde Nov 3, 2024
2af2b3f
Ad a bit more controll to synchronizations
rubenvdlinde Nov 3, 2024
361763d
Merge remote-tracking branch 'origin/development' into fix/small-fixe…
rubenvdlinde Nov 4, 2024
4eeb910
Merge remote-tracking branch 'origin/feature/CONNECTOR-62/synchroniza…
rubenvdlinde Nov 4, 2024
5b5bbd6
Merge remote-tracking branch 'origin/feature/CONNECTOR-48/synchro-tes…
rubenvdlinde Nov 5, 2024
f0489ce
switches from originIdLocation to idPosition on source config
rubenvdlinde Nov 5, 2024
1b28cbc
Still testing the sync flow
rubenvdlinde Nov 5, 2024
42ec47f
Fixed conflicting true/false
rubenvdlinde Nov 5, 2024
a97e512
Remove var dumps
rubenvdlinde Nov 5, 2024
ec422d2
Merge dev
bbrands02 Nov 5, 2024
ca3f378
Pagination improvements
bbrands02 Nov 5, 2024
be34bb9
Verwerken feedback Wilco
rubenvdlinde Nov 6, 2024
61effa4
Merge remote-tracking branch 'origin/feature/CONNECTOR-62/synchroniza…
rubenvdlinde Nov 6, 2024
150d854
CallService add pagination from config
bbrands02 Nov 6, 2024
63b31d7
Cleanup and pagination fixes
bbrands02 Nov 6, 2024
e854490
Making the mapping service ussable for other nextcloud applications
rubenvdlinde Nov 7, 2024
98c7289
Declared todo
rubenvdlinde Nov 7, 2024
26e2242
WIP, made sync logs viewable
remko48 Nov 7, 2024
372f430
Merge remote-tracking branch 'origin/fix/small-fixes-based-on-loging'…
remko48 Nov 7, 2024
a1c9292
Fix infinite loop
rjzondervan Nov 7, 2024
d795c10
small style fix
bbrands02 Nov 7, 2024
e938fea
style improvements
bbrands02 Nov 7, 2024
9789287
More fixes on infinite loop
rjzondervan Nov 7, 2024
6ab6ce9
Locational cleanup
rjzondervan Nov 7, 2024
41ef79a
Pagination improvement
bbrands02 Nov 7, 2024
e350372
Fix reintroduced deadloop
rjzondervan Nov 7, 2024
871d59e
Code comment added
bbrands02 Nov 7, 2024
e02d8f7
Show logs fix
remko48 Nov 7, 2024
a17dfbd
Added log templates
remko48 Nov 7, 2024
c8be074
Merge remote-tracking branch 'origin/development' into fix/small-fixe…
rjzondervan Nov 7, 2024
791dc23
fixed objects mapping
RalkeyOfficial Nov 7, 2024
36e3165
Shorter function name
WilcoLouwerse Nov 7, 2024
88493bc
Quick cleanup SynchronizationService
WilcoLouwerse Nov 7, 2024
d8c9331
Simple fix for unclear if statement
WilcoLouwerse Nov 7, 2024
abe7cd4
Merge pull request #46 from ConductionNL/fix/small-fixes-based-on-loging
WilcoLouwerse Nov 7, 2024
d40ed45
Merge pull request #50 from ConductionNL/feature/REGISTERS-47/objects…
WilcoLouwerse Nov 7, 2024
b8c6c74
fixed mapping test cast and made it optional
RalkeyOfficial Nov 8, 2024
ccc4dbe
Merge pull request #51 from ConductionNL/feature/CONNECTOR-118/cast-fix
remko48 Nov 8, 2024
14fa9ef
hotfix synchronization modal
RalkeyOfficial Nov 8, 2024
263c6aa
Merge pull request #53 from ConductionNL/hofix/synchronization-modal
remko48 Nov 8, 2024
e7c9e28
added a valid or invalid text with an inline icon
RalkeyOfficial Nov 8, 2024
7abe296
Merge branch 'development' into feature/CONNECTOR-121/mapping-test-re…
RalkeyOfficial Nov 11, 2024
4fc0231
finished mapping test result looks
RalkeyOfficial Nov 11, 2024
b592bf4
small bug fix
RalkeyOfficial Nov 11, 2024
f38624e
wrong type of bug fix
RalkeyOfficial Nov 11, 2024
38fbfb7
fixed a issue with validation errors being undefined
RalkeyOfficial Nov 11, 2024
59494c9
Fix migrations for merge
WilcoLouwerse Nov 11, 2024
9a0db8e
Improve if statements in latest migration
WilcoLouwerse Nov 11, 2024
09eff98
Fix typo in if statement latest migration
WilcoLouwerse Nov 11, 2024
5d2f61f
Use changeColumn instead of renameColumn (=doesn't exist @chatgpt...)
WilcoLouwerse Nov 11, 2024
af3ad80
fixed console error
RalkeyOfficial Nov 11, 2024
48e5a16
Final fix for renaming columns in the last migration while keeping data
WilcoLouwerse Nov 11, 2024
668905e
Merge pull request #56 from ConductionNL/feature/CONNECTOR-129/delete…
RalkeyOfficial Nov 12, 2024
1bbe6b7
Merge pull request #54 from ConductionNL/feature/CONNECTOR-121/mappin…
RalkeyOfficial Nov 12, 2024
9571b72
Merge branch 'development' into fix/migrations-for-merge
WilcoLouwerse Nov 12, 2024
f705cb0
Keep old properties
bbrands02 Nov 12, 2024
06a1aff
Merge pull request #58 from ConductionNL/fix/old-sync-contract-fields
WilcoLouwerse Nov 12, 2024
d050c23
Update migrations
WilcoLouwerse Nov 12, 2024
c2b4c44
Merge branch 'development' into fix/migrations-for-merge
WilcoLouwerse Nov 12, 2024
e3f688c
Add if statement just in case
WilcoLouwerse Nov 12, 2024
693fa3d
Remove the extra migration that broke things
WilcoLouwerse Nov 12, 2024
ec9854b
Merge pull request #55 from ConductionNL/fix/migrations-for-merge
WilcoLouwerse Nov 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions appinfo/routes.php
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
['name' => 'endpoints#logs', 'url' => '/api/endpoints-logs/{id}', 'verb' => 'GET'],
['name' => 'synchronizations#contracts', 'url' => '/api/synchronizations-contracts/{id}', 'verb' => 'GET'],
['name' => 'synchronizations#logs', 'url' => '/api/synchronizations-logs/{id}', 'verb' => 'GET'],
['name' => 'synchronizations#test', 'url' => '/api/synchronizations-test/{id}', 'verb' => 'POST'],
// Mapping endpoints
['name' => 'mappings#test', 'url' => '/api/mappings/test', 'verb' => 'POST'],
// Running endpoints
Expand Down
34 changes: 9 additions & 25 deletions lib/Action/SynchronizationAction.php
Original file line number Diff line number Diff line change
Expand Up @@ -44,42 +44,25 @@ public function __construct(
*/
public function run(array $argument = []): array
{

$response = [];

// if we do not have a synchronization Id then everything is wrong
$response['stackTrace'][] = 'Check for a valid synchronization ID';
if (!isset($argument['synchronizationId'])) {
$response['message'] = $response['stackTrace'][] = 'Check for a valid synchronization ID';
if (isset($argument['synchronizationId']) === false) {
// @todo: implement error handling
$response['level'] = 'ERROR';
$response['message'] = 'No synchronization ID provided';
return $response;
}
$response['stackTrace'][] = $response['message'] = 'No synchronization ID provided';

// We are going to allow for a single synchronization contract to be processed at a time
if (isset($argument['synchronizationContractId']) && is_int((int) $argument['synchronizationContractId'])) {
$response['level'] = 'INFO';
$response['message'] = 'Synchronization single contract: '.$argument['synchronizationContractId'];
$synchronizationContract = $this->synchronizationContractMapper->find((int) $argument['synchronizationContractId']);
if($synchronizationContract === null){
$response['level'] = 'ERROR';
$response['message'] = 'Contract not found: '.$argument['synchronizationContractId'];
return $response;
}
try {
$this->callService->synchronizeContract($synchronization);
} catch (Exception $e) {
$response['level'] = 'ERROR';
$response['message'] = 'Failed to synchronize contract: ' . $e->getMessage();
return $response;
}
return $response;
}

// Let's find a synchronysation
$response['stackTrace'][] = 'Getting synchronization: '.$argument['synchronizationId'];
$synchronization = $this->synchronizationMapper->find((int) $argument['synchronizationId']);
if ($synchronization === null){
$response['level'] = 'WARNING';
$response['message'] = 'Synchronization not found: '.$argument['synchronizationId'];
$response['stackTrace'][] = $response['message'] = 'Synchronization not found: '.$argument['synchronizationId'];
return $response;
}

Expand All @@ -89,11 +72,12 @@ public function run(array $argument = []): array
$objects = $this->synchronizationService->synchronize($synchronization);
} catch (Exception $e) {
$response['level'] = 'ERROR';
$response['message'] = 'Failed to synchronize: ' . $e->getMessage();
$response['stackTrace'][] = $response['message'] = 'Failed to synchronize: ' . $e->getMessage();
return $response;
}

$response['stackTrace'][] = 'Synchronized '.count($objects).' successfully';
$response['level'] = 'INFO';
$response['stackTrace'][] = $response['message'] = 'Synchronized '.count($objects).' successfully';

// Let's report back about what we have just done
return $response;
Expand Down
31 changes: 25 additions & 6 deletions lib/Controller/JobsController.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

namespace OCA\OpenConnector\Controller;

use Exception;
use OCA\OpenConnector\Service\ObjectService;
use OCA\OpenConnector\Service\SearchService;
use OCA\OpenConnector\Db\Job;
Expand All @@ -14,6 +15,10 @@
use OCP\BackgroundJob\IJobList;
use OCA\OpenConnector\Db\JobLogMapper;
use OCA\OpenConnector\Service\JobService;
use OCP\AppFramework\Db\DoesNotExistException;

use OCA\OpenConnector\Service\SynchronizationService;
use OCA\OpenConnector\Db\SynchronizationMapper;

class JobsController extends Controller
{
Expand All @@ -32,10 +37,12 @@ public function __construct(
private JobLogMapper $jobLogMapper,
private JobService $jobService,
private IJobList $jobList,
private SynchronizationService $synchronizationService,
private SynchronizationMapper $synchronizationMapper
)
{
parent::__construct($appName, $request);
$this->IJobList = $jobList;
$this->jobList = $jobList;
}

/**
Expand Down Expand Up @@ -219,13 +226,25 @@ public function run(int $id): JSONResponse
{
try {
$job = $this->jobMapper->find(id: $id);
if (!$job->getJobListId()) {
return new JSONResponse(data: ['error' => 'Job not scheduled'], statusCode: 404);
}
$this->IJobList->getById($job->getJobListId())->start($this->IJobList);
return new JSONResponse($this->jobLogMapper->getLastCallLog());
} catch (DoesNotExistException $exception) {
return new JSONResponse(data: ['error' => 'Not Found'], statusCode: 404);
}


if ($job->getJobListId() === false) {
return new JSONResponse(data: ['error' => 'Job not scheduled'], statusCode: 404);
}

try {
$this->jobList->getById($job->getJobListId())->start($this->jobList);
$lastLog = $this->jobLogMapper->getLastCallLog();
if ($lastLog !== null) {
return new JSONResponse(data: $lastLog, statusCode: 200);
}

return new JSONResponse(data: ['error' => 'No job log could be found, job did not went succesfully or failed to log anything'], statusCode: 500);
} catch (Exception $exception) {
return new JSONResponse(data: ['error' => $exception->getMessage()], statusCode: 400);
}
}
}
84 changes: 69 additions & 15 deletions lib/Controller/SynchronizationsController.php
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

use OCA\OpenConnector\Service\ObjectService;
use OCA\OpenConnector\Service\SearchService;
use OCA\OpenConnector\Db\Synchronization;
use OCA\OpenConnector\Service\SynchronizationService;
use OCA\OpenConnector\Db\SynchronizationMapper;
use OCA\OpenConnector\Db\SynchronizationContractMapper;
use OCA\OpenConnector\Db\SynchronizationContractLogMapper;
Expand All @@ -13,6 +13,8 @@
use OCP\AppFramework\Http\JSONResponse;
use OCP\IAppConfig;
use OCP\IRequest;
use Exception;
use OCP\AppFramework\Db\DoesNotExistException;

class SynchronizationsController extends Controller
{
Expand All @@ -29,16 +31,17 @@ public function __construct(
private readonly IAppConfig $config,
private readonly SynchronizationMapper $synchronizationMapper,
private readonly SynchronizationContractMapper $synchronizationContractMapper,
private readonly SynchronizationContractLogMapper $synchronizationContractLogMapper
private readonly SynchronizationContractLogMapper $synchronizationContractLogMapper,
private readonly SynchronizationService $synchronizationService
)
{
parent::__construct($appName, $request);
parent::__construct($appName, $request);

}

/**
* Returns the template of the main app's page
*
*
* This method renders the main page of the application, adding any necessary data to the template.
*
* @NoAdminRequired
Expand All @@ -47,17 +50,17 @@ public function __construct(
* @return TemplateResponse The rendered template response
*/
public function page(): TemplateResponse
{
{
return new TemplateResponse(
'openconnector',
'index',
[]
);
}

/**
* Retrieves a list of all synchronizations
*
*
* This method returns a JSON response containing an array of all synchronizations in the system.
*
* @NoAdminRequired
Expand All @@ -79,7 +82,7 @@ public function index(ObjectService $objectService, SearchService $searchService

/**
* Retrieves a single synchronization by its ID
*
*
* This method returns a JSON response containing the details of a specific synchronization.
*
* @NoAdminRequired
Expand All @@ -99,7 +102,7 @@ public function show(string $id): JSONResponse

/**
* Creates a new synchronization
*
*
* This method creates a new synchronization based on POST data.
*
* @NoAdminRequired
Expand All @@ -116,17 +119,17 @@ public function create(): JSONResponse
unset($data[$key]);
}
}

if (isset($data['id'])) {
unset($data['id']);
}

return new JSONResponse($this->synchronizationMapper->createFromArray(object: $data));
}

/**
* Updates an existing synchronization
*
*
* This method updates an existing synchronization based on its ID.
*
* @NoAdminRequired
Expand All @@ -152,7 +155,7 @@ public function update(int $id): JSONResponse

/**
* Deletes a synchronization
*
*
* This method deletes a synchronization based on its ID.
*
* @NoAdminRequired
Expand Down Expand Up @@ -182,7 +185,7 @@ public function destroy(int $id): JSONResponse
public function contracts(int $id): JSONResponse
{
try {
$contracts = $this->synchronizationContractMapper->findAll($null, null, ['synchronization_id' => $id]);
$contracts = $this->synchronizationContractMapper->findAll(null, null, ['synchronization_id' => $id]);
return new JSONResponse($contracts);
} catch (DoesNotExistException $e) {
return new JSONResponse(['error' => 'Contracts not found'], 404);
Expand All @@ -199,7 +202,7 @@ public function contracts(int $id): JSONResponse
*
* @param int $id The ID of the source to retrieve logs for
* @return JSONResponse A JSON response containing the call logs
*/
*/
public function logs(int $id): JSONResponse
{
try {
Expand All @@ -209,4 +212,55 @@ public function logs(int $id): JSONResponse
return new JSONResponse(['error' => 'Logs not found'], 404);
}
}

/**
* Tests a synchronization
*
* This method tests a synchronization without persisting anything to the database.
*
* @NoAdminRequired
* @NoCSRFRequired
*
* @param int $id The ID of the synchronization
*
* @return JSONResponse A JSON response containing the test results
*
* @example
* Request:
* empty POST
*
* Response:
* {
* "resultObject": {
* "fullName": "John Doe",
* "userAge": 30,
* "contactEmail": "[email protected]"
* },
* "isValid": true,
* "validationErrors": []
* }
*/
public function test(int $id): JSONResponse
{
try {
$synchronization = $this->synchronizationMapper->find(id: $id);
} catch (DoesNotExistException $exception) {
return new JSONResponse(data: ['error' => 'Not Found'], statusCode: 404);
}

// Try to synchronize
try {
$logAndContractArray = $this->synchronizationService->synchronize(synchronization: $synchronization, isTest: true);
// Return the result as a JSON response
return new JSONResponse(data: $logAndContractArray, statusCode: 200);
} catch (Exception $e) {
// If synchronizaiton fails, return an error response
return new JSONResponse([
'error' => 'Synchronization error',
'message' => $e->getMessage()
], 400);
}

return new JSONResponse($resultFromTest, 200);
}
}
Loading