Investigate using LLM (e.g. ChapGPT) as a Processor

[chadell]

Proposed Functionality

New LLM (Large Language Model) allow a sophisticated language processing that could get data from structures/unstructured notifications and retrieve part or total of the necessary information.

Use Case

This could avoid having to write custom parser for all Providers

[jvanderaa]

I think this is a great idea. I was also thinking through the idea of send it to a LLM when parsing fails. So allow for a conditional.

[chadell]

Yes, my idea would be to make it the last resort when the known parser failed or there is no parser at all.
Preliminary testing looks good!

[KedharnathGoud]

We can use something like https://github.com/gventuri/pandas-ai

[AndriusV4]

Hi, this idea would be amazing if implemented. Was wondering is this still in testing phase?
I have multiple providers that are not supported by the library, so just checking if I should start writing customer parsers for it or are there any plans to implement LLM parser in the near future that could help with this?

Thanks!

[chadell]

Hi @AndriusV4 , this is something we definitely want to explore, but we have not started yet.
Any contribution in this direction would be welcome!

[chadell]

I already have PoC that allows parsing only using Machine Learning:

$ circuit-maintenance-parser --data-file "tests/unit/data/aws/aws1.eml" --data-type email 

Circuit Maintenance Notification #0
{
  "account": "aaaaa",
  "circuits": [
    {
      "circuit_id": "aaaaa-00000001",
      "impact": "OUTAGE"
    },
    {
      "circuit_id": "aaaaa-00000002",
      "impact": "OUTAGE"
    },
    {
      "circuit_id": "aaaaa-00000003",
      "impact": "OUTAGE"
    },
    {
      "circuit_id": "aaaaa-00000004",
      "impact": "OUTAGE"
    },
    {
      "circuit_id": "aaaaa-00000005",
      "impact": "OUTAGE"
    },
    {
      "circuit_id": "aaaaa-00000006",
      "impact": "OUTAGE"
    }
  ],
  "end": 1621522800,
  "maintenance_id": "d9f256dcae6c34a4d7c353b19f02957c",
  "organizer": "unknown",
  "provider": "genericprovider",
  "sequence": 1,
  "stamp": 1620337976,
  "start": 1621497600,
  "status": "CONFIRMED",
  "summary": "Planned maintenance has been scheduled on an AWS Direct Connect router in A Block, New York, NY from Thu, 20 May 2021 08:00:00 GMT to Thu, 20 May 2021 14:00:00 GMT for 6 hours.",
  "uid": "0"
}

[chadell]

done in #245