Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Improve LGBM exception and logging #2037

Merged
merged 2 commits into from
Aug 2, 2023
Merged

Conversation

svotaw
Copy link
Collaborator

@svotaw svotaw commented Jul 31, 2023

Fix a bug in exception handling and improve logs for LightGBM

@github-actions
Copy link

Hey @svotaw 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

  • fix: Fix LightGBM crashes with empty partitions
  • feat: Make HTTP on Spark back-offs configurable
  • docs: Update Spark Serving usage
  • build: Add codecov support
  • perf: improve LightGBM memory usage
  • refactor: make python code generation rely on classes
  • style: Remove nulls from CNTKModel
  • test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary by GPT-4

The changes in this commit include:

  1. In NetworkManager.scala, the log message for "received nodes for network init" has been moved after the log message for "received partition topology". This change makes the log messages appear in a more logical order.

  2. A new log message has been added in parseExecutorPartitionList function to show the partitions received by each executor.

  3. In StreamingPartitionTask.scala, some log messages have been changed from info level to debug level, as they might be too verbose for normal operation. These messages include "Inserting rows into training Dataset" and "Pushing dense/sparse rows".

  4. Some unnecessary log messages have been removed, such as "Adjusting initial scores", "Pushing sparse rows", and "LightGBM pushed startIndex in partition".

Overall, these changes improve the readability and usefulness of the log messages generated by LightGBM during training.

Suggestions

No suggestions are needed as the changes in this PR seem appropriate.

@svotaw svotaw changed the title Improve LGBM exception and logging fix: Improve LGBM exception and logging Jul 31, 2023
@svotaw
Copy link
Collaborator Author

svotaw commented Jul 31, 2023

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@codecov-commenter
Copy link

codecov-commenter commented Jul 31, 2023

Codecov Report

Merging #2037 (70a38c8) into master (db6386c) will decrease coverage by 1.37%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##           master    #2037      +/-   ##
==========================================
- Coverage   87.09%   85.72%   -1.37%     
==========================================
  Files         306      306              
  Lines       16068    16063       -5     
  Branches      860      852       -8     
==========================================
- Hits        13994    13770     -224     
- Misses       2074     2293     +219     
Files Changed Coverage Δ
...oft/azure/synapse/ml/lightgbm/NetworkManager.scala 92.26% <75.00%> (+0.04%) ⬆️
...e/synapse/ml/lightgbm/StreamingPartitionTask.scala 97.02% <100.00%> (-0.11%) ⬇️

... and 8 files with indirect coverage changes

@svotaw svotaw merged commit cde6834 into microsoft:master Aug 2, 2023
67 of 69 checks passed
JessicaXYWang pushed a commit to JessicaXYWang/SynapseML that referenced this pull request Sep 14, 2023
* Improve LGBM exception and logging

* added log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants