Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Signaling the Data Source that has a problem #2755

Closed
32 tasks done
Limraj opened this issue Nov 2, 2023 · 0 comments
Closed
32 tasks done

Signaling the Data Source that has a problem #2755

Limraj opened this issue Nov 2, 2023 · 0 comments
Assignees
Milestone

Comments

@Limraj
Copy link
Collaborator

Limraj commented Nov 2, 2023

Expected Behavior
If the Data Source is having script executing/connections problems, this should be visible to the user in watchlists and graphical views.

To reproduce

  1. Login as admin;
  2. Create Virtual Data Source as virtual_ds (enabled, update: 1s) and Virtual Data Point as virtual_dp_1 (enabled, numeric, random, min:0, max:123, start value:0) and virtual_dp_2 (enabled, settable, binary, no change, start value:0)
  3. Create Meta Data Source as meta_ds (enabled) and Meta Data Point as meta_dp (enabled, numeric) Added to context:
    p1 - 'virtual_ds - virtual_dp_1';
    p2 - 'virtual_ds - virtual_dp_2';

    and set script:
    if(p2.value) return p1.value; return -1;
  4. Added to watchlist as wl_1, add added points:
    a) 'meta_ds - meta_dp';
    b) 'virtual_ds - virtual_dp_1';
    c) 'virtual_ds - virtual_dp_2';
  5. Disabled virtual_ds;
  6. meta_dp does not change and does not indicate a problem -> ERROR 1;
  7. Enabled virtaul ds -> meta_ds meta_dp value changes as expected;
  8. Deleted 'virtual_ds - virtual_dp_1' -> identical behavior as in step 7, meta_dp does not indicate a problem -> ERROR 2;

Test procedure:

  1. Case from issue to check, you can check it first on the old code;

  2. You import a new configuration and check how it works, if you disable some virtual data point, or sql data point, or all of them, are you able to find out from the alarms that these meta data points do not work because you have disabled points in their context and specifically which points are in question, which specific points are causing the problem;
    Sign (2).zip

  3. You configure your own structure, let it be for these 2 nestings and repeat the tests;

  4. You create a simple cyclic configuration meta_1 -> meta_2 > meta_1, and meta_1 -> meta_2 -> meta_3 -> meta_1

  5. You create as many of these Data Sources with points as possible and check if there is no generation of multiple events;

To do
It is worth adding such signaling at least in:

  • Meta Data Source
  • Modbus IP
  • SNMP
  • HTTP Retriever
  • SQL
  • MQTT
  • Internal Data Source
  • Virtual Data Source
  • JMX
  • Modbus Serial
  • 1-wire
  • BACnet I/P
  • DNP3 IP
  • DNP3 Serial
  • AMQP
  • ASCII File Reader
  • ASCII Serial
  • Dr.Storage HT-5B
  • Galil DMC-21x2
  • HTTP Image
  • HTTP Receiver
  • IEC101 Ethernet
  • IEC101 Serial
  • M Bus
  • Mitsubishi Alpha2
  • NMEA listener
  • OPC DA
  • OpenV4J
  • Pachube
  • POP3 Email
  • Serotonin Persistent TCP
  • VMStat Data Source

You can follow the solution included in Data Source MQTT/AMQP;

Specifications
Scada-LTS Version: 2.6.18

Linked:
#1745

@Limraj Limraj added the bug label Nov 2, 2023
@Limraj Limraj added this to the 2.7.7 milestone Nov 2, 2023
Limraj added a commit that referenced this issue Nov 6, 2023
Created class: DataPointUnreliableUtils;

Signaling added in:
*  Meta Data Source;
*  SNMP;
*  HTTP Retriever;
*  BACnet I/P;
*  DNP3 (IP/Serial);

Signaling use DataPointUnreliableUtils in:
* Modbus IP/Serial;
* Viconics;
* Pachube;
* MQTT/AMQP
Limraj added a commit that referenced this issue Nov 6, 2023
Limraj added a commit that referenced this issue Nov 7, 2023
@Limraj Limraj self-assigned this Nov 13, 2023
@Limraj Limraj modified the milestones: 2.7.7, 2.7.8 Nov 17, 2023
@Limraj Limraj modified the milestones: 2.7.8, 2.7.7 Jan 30, 2024
Limraj added a commit that referenced this issue Feb 1, 2024
- fixed feature refresh point value
Limraj added a commit that referenced this issue Feb 9, 2024
- DataPointUnreliableUtilsTest.java;
- use methods returnToNormal and raiseEvent from DataSourceRT;
- added methods: LoggingUtils.info, LoggingUtils.dataPointInfo, LoggingUtils.dataPointRtInfo, PollingDataSource.getDataPoints, EventDataSource.getDataPoints, DataSourceRT.getDataPoints;
Limraj added a commit that referenced this issue Feb 21, 2024
- fixed Data Source use serial port - System Exception;
Limraj added a commit that referenced this issue Feb 27, 2024
- added tests: InitializeDataSourceRtTest, InitializeWithErrorsDataSourceRtTest;
- corrected test: DataPointUnreliableUtilsTest;
- replacing class CommPortIdentifier from RXTXcomm.jar with a new class ScadaCommPortIdentifier: ASCIISerialDataSource, DrStorageHt5bDataSource, RadiuinoEventDataSource, RadiuinoPollingDataSource;
- corrected signaling errors in: BACnetIPDataSourceRT, HttpReceiverDataSourceRT, ModbusDataSource, NmeaDataSourceRT, PachubeDataSourceRT, PersistentDataSourceRT, SnmpDataSourceRT, ViconicsDataSourceRT;
- try catch Throwable;
- change logging level from WARN to ERROR for dataSourceRtLoggingLevel, dataPointRtLoggingLevel (log4j2.xml);
- refactoring DataPointUnreliableUtils;
- added methods: DataPointRT.isUnreliable, DataPointRT.isSetUnreliable, DataSourceUtils.checkInitialized;
Limraj added a commit that referenced this issue Feb 27, 2024
- corrected test: InitializeWithErrorsDataSourceRtTest;
- change type catch from IOException to Throwable;
Limraj added a commit that referenced this issue Feb 27, 2024
- corrected test: InitializeWithErrorsDataSourceRtTest for VMStatDataSourceRT - update PrepareForTest annotation, if dataSourceRT instanceof VMStatDataSourceRT, for MessagingDataSourceRT with MQTT throw MessagingServiceException after invoke MessageService.open;
Limraj added a commit that referenced this issue Feb 28, 2024
- fixed generate many events by ModbusDataSource;
- corrected test InitializeDataSourceRtTest;
- refactor InitializeDataSourceRtMockUtils;
Limraj added a commit that referenced this issue Feb 28, 2024
- added class: RuntimeMockUtils
Limraj added a commit that referenced this issue Feb 28, 2024
- fixed generate many events by: Pop3DataSourceRT, PachubeDataSourceRT, NmeaDataSourceRT, HttpRetrieverDataSourceRT, BACnetIPDataSourceRT, change to active events;
- change logging fatal to debug level in methods: MBusDataSourceRT.initialize(), MBusDataSourceRT.terminate();
- fixed DataPointUnreliableUtilsTest, InitializeDataSourceRtTest, InitializeWithErrorsDataSourceRtTest, mocking TimeoutTask;
Limraj added a commit that referenced this issue Feb 28, 2024
- corrected terminate method for: Alpha2DataSource, Dnp3DataSource, DrStorageHt5bDataSource, IEC101DataSource, OPCDataSource, RadiuinoEventDataSource, RadiuinoPollingDataSource;
- fixed reset unreliable HttpImageDataSourceRT when error retriever;
- refactor DataPointUnreliableUtils;
- fixed RuntimeManager.startDataPoint;
Limraj added a commit that referenced this issue Feb 28, 2024
- added tests: InitializeDataSourceRtTest, InitializeWithErrorsDataSourceRtTest;
- corrected test: DataPointUnreliableUtilsTest;
- replacing class CommPortIdentifier from RXTXcomm.jar with a new class ScadaCommPortIdentifier: ASCIISerialDataSource, DrStorageHt5bDataSource, RadiuinoEventDataSource, RadiuinoPollingDataSource;
- corrected signaling errors in: BACnetIPDataSourceRT, HttpReceiverDataSourceRT, ModbusDataSource, NmeaDataSourceRT, PachubeDataSourceRT, PersistentDataSourceRT, SnmpDataSourceRT, ViconicsDataSourceRT;
- try catch Throwable;
- change logging level from WARN to ERROR for dataSourceRtLoggingLevel, dataPointRtLoggingLevel (log4j2.xml);
- refactoring DataPointUnreliableUtils;
- added methods: DataPointRT.isUnreliable, DataPointRT.isSetUnreliable, DataSourceUtils.checkInitialized;
Limraj added a commit that referenced this issue Feb 28, 2024
- corrected test: InitializeWithErrorsDataSourceRtTest;
- change type catch from IOException to Throwable;
Limraj added a commit that referenced this issue Feb 28, 2024
- corrected test: InitializeWithErrorsDataSourceRtTest for VMStatDataSourceRT - update PrepareForTest annotation, if dataSourceRT instanceof VMStatDataSourceRT, for MessagingDataSourceRT with MQTT throw MessagingServiceException after invoke MessageService.open;
Limraj added a commit that referenced this issue Feb 28, 2024
- fixed generate many events by ModbusDataSource;
- corrected test InitializeDataSourceRtTest;
- refactor InitializeDataSourceRtMockUtils;
Limraj added a commit that referenced this issue Feb 28, 2024
- added class: RuntimeMockUtils
Limraj added a commit that referenced this issue Feb 28, 2024
- fixed generate many events by: Pop3DataSourceRT, PachubeDataSourceRT, NmeaDataSourceRT, HttpRetrieverDataSourceRT, BACnetIPDataSourceRT, change to active events;
- change logging fatal to debug level in methods: MBusDataSourceRT.initialize(), MBusDataSourceRT.terminate();
- fixed DataPointUnreliableUtilsTest, InitializeDataSourceRtTest, InitializeWithErrorsDataSourceRtTest, mocking TimeoutTask;
Limraj added a commit that referenced this issue Feb 28, 2024
- fixed generate many events by SqlDataSourceRT;
Limraj added a commit that referenced this issue Feb 29, 2024
- fixed export DrStorageHt5bDataSourceVO by removed @JsonRemoteProperty from validate method;
@Limraj Limraj modified the milestones: 2.7.7, 2.7.7.1, 2.7.8 Feb 29, 2024
@Limraj Limraj modified the milestones: 2.7.8, 2.8.0 May 16, 2024
Limraj added a commit that referenced this issue Jul 13, 2024
- fixed updateAttemptsCounters;
- test.failFast, testLogging.exceptionFormat = full;
Limraj added a commit that referenced this issue Aug 6, 2024
- Corrected config log4j2.xml for script: ScriptExecutor, ContextualizedScriptRT;
- Added new event type DataSourcePointEventType;
- Added migration V2_8_0_1__AddTypeRef3ColumnToEvents;
Limraj added a commit that referenced this issue Aug 30, 2024
- fixed alarmAck.tag for new event type;
- fixed double actions for eventList.jsp;
Limraj added a commit that referenced this issue Sep 4, 2024
- fixed url: scripting.jsp, dataPointEdit.jsp, systemSettings.jsp;
- corrected MetaPointLocatorRT - reduce generate events, DataSourceEditDwr, PointValueService, ContextualizedScriptRT - catch Exception;
- fixed ScriptExecutor, event per point;
- set IGNORE_SAME_MESSAGE for DataSourcePointEventType.duplicateHandling;
Limraj added a commit that referenced this issue Sep 6, 2024
- fixed ASCIIFileDataSource, BACnetIPDataSourceRT;
Limraj added a commit that referenced this issue Sep 6, 2024
- set ERROR logging level for dataSourceRtLoggingLevel
Limraj added a commit that referenced this issue Sep 18, 2024
…Source_that_has_a_problem

Fix/#2755 signaling the data source that has a problem
@Limraj Limraj closed this as completed Sep 18, 2024
Limraj added a commit that referenced this issue Oct 23, 2024
- refactoring V2_8_0_1__AddTypeRef3ColumnToEvents;
@Limraj Limraj reopened this Oct 27, 2024
Limraj added a commit that referenced this issue Nov 6, 2024
…Source_that_has_a_problem

#2755 Signaling the data source that has a problem:
@Limraj Limraj closed this as completed Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant