Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Error during classification analytics #1040

Closed
pheyos opened this issue Mar 6, 2020 · 1 comment · Fixed by #1042
Closed

[ML] Error during classification analytics #1040

pheyos opened this issue Mar 6, 2020 · 1 comment · Fixed by #1042

Comments

@pheyos
Copy link
Member

pheyos commented Mar 6, 2020

Found in version

Steps to reproduce

  • Create and start a classification job based in the bank-marketing dataset
  • Configuration:
{
  "description": "",
  "source": {
    "index": "bank-marketing*"
  },
  "dest": {
    "index": "dest-bm_1"
  },
  "analyzed_fields": {
    "excludes": []
  },
  "analysis": {
    "classification": {
      "dependent_variable": "y",
      "training_percent": "20"
    }
  },
  "model_memory_limit": "200mb"
}

Expected result

  • The classification job finishes without errors

Actual result

  • The job goes into failed state with the following error in the job messages:
Updated analytics task state to [failed] with reason [[bm_1_1583497643615] analytics process stopped unexpectedly: Fatal error: 'si_signo 11, si_code: 1, si_errno: 0, address: 0x7ff15024cc91, library: /home/robert/github/machine-learning-qa/build/deployments/8.0.0/6683/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/../lib/libMlMaths.so, base: 0x7ff150015000, normalized address: 0x237c91', version: 8.0.0-SNAPSHOT (build 9abec24aff83ac) ]

Additional information

  • When running a classification job with different input data, e.g. Kibana sample flight data, the job runs without errors, so it's probably input data related.
@pheyos pheyos added the >bug label Mar 6, 2020
@droberts195
Copy link
Contributor

droberts195 commented Mar 6, 2020

There was a log message capturing the failure address:

Updated analytics task state to [failed] with reason [[bm_1_1583489004749] analytics process stopped unexpectedly:
Fatal error: 'si_signo 11, si_code: 1, si_errno: 0, address: 0x7f8968223941, library: /home/robert/github/machine-learning-qa/build/deployments/8.0.0/6687/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/../lib/libMlMaths.so, base: 0x7f8967feb000, normalized address: 0x238941', version: 8.0.0-SNAPSHOT (build 0108a6ecf8a983) ]

Looking up the function from that shows the crash happened in _ZN2ml5maths15CDataFrameUtils19metricMicWithColumnERKNS1_12CColumnValueERKNS_4core10CDataFrameERKNS5_16CPackedBitVectorESt6vectorImSaImEE, which demangles to ml::maths::CDataFrameUtils::metricMicWithColumn(ml::maths::CDataFrameUtils::CColumnValue const&, ml::core::CDataFrame const&, ml::core::CPackedBitVector const&, std::vector<unsigned long, std::allocator<unsigned long> >).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants