Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added feature store and pyspark notebooks with sagemaker_core SDK #4748

Open
wants to merge 5 commits into
base: default
Choose a base branch
from
Open

Added feature store and pyspark notebooks with sagemaker_core SDK #4748

wants to merge 5 commits into from

Conversation

A-yush
Copy link
Contributor

@A-yush A-yush commented Sep 6, 2024

Feature store notebook is giving error due to pydantic library. Pyspark processing notebook is working fine and I have updated the code according to sagemaker_core SDK

Issue #, if available: Feature store group creation not working due to pydantic error. Pyspark processing notebook is working as expected. Modified code according to sagemaker_core

Description of changes: Added feature store and pyspark notebooks with sagemaker_core S

Testing done: Yes for Pyspark processing notebook

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

  • [ X] I have verified that my PR does not contain any new notebook/s which demonstrate a SageMaker functionality already showcased by another existing notebook in the repository
  • [X ] I have read the CONTRIBUTING doc and adhered to the guidelines regarding folder placement, notebook naming convention and example notebook best practices
  • I have updated the necessary documentation, including the README of the appropriate folder as well as the index.rst file
  • [X ] I have tested my notebook(s) and ensured it runs end-to-end
  • I have linted my notebook(s) and code using black-nb -l 100 {path}/{notebook-name}.ipynb

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@A-yush
Added feature store and pyspark notebooks
38386e6 
Feature store notebook is giving error due to pydantic library. Pyspark processing notebook is working fine and I have updated the code according to sagemaker_core SDK
@review-notebook-app Review Notebook App
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-amazon-sagemaker-examples-pr
  • Commit ID: 38386e6
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-code-formatting
  • Commit ID: 38386e6
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-link-check
  • Commit ID: 38386e6
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

pintaoz-aws
pintaoz-aws reviewed Sep 11, 2024
View reviewed changes
sagemaker-core/spark-processing-sm-core.ipynb
@@ -0,0 +1,1059 @@
{
Copy link
Contributor

@pintaoz-aws pintaoz-aws Sep 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: clear the cell outputs


Reply via ReviewNB

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

sagemaker-core/spark-processing-sm-core.ipynb
@@ -0,0 +1,1059 @@
{
Copy link
Contributor

@pintaoz-aws pintaoz-aws Sep 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to set region here, we can let users to use their default region


Reply via ReviewNB

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

sagemaker-core/spark-processing-sm-core.ipynb
@@ -0,0 +1,1059 @@
{
Copy link
Contributor

@pintaoz-aws pintaoz-aws Sep 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a cell to delete the resources created in this notebook at the end?


Reply via ReviewNB

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

processing jobs once created can't be deleted. Once they are completed, costs stop getting incurred.

@A-yush
added modified spark-processing-sm-core.ipynb
ccc1759 
removed outputs, used default bucket only and wrote some comments for verbosity
@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-amazon-sagemaker-examples-pr
  • Commit ID: ccc1759
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-link-check
  • Commit ID: ccc1759
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-code-formatting
  • Commit ID: ccc1759
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@A-yush
modified pyspark processing notebook
033e5a4 
removed outputs, used default s3 bucket and added additional comments for verbosity
@A-yush
Merge pull request #2 from A-yush/A-yush-patch-2
884cba8 
modified pyspark processing notebook
@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-amazon-sagemaker-examples-pr
  • Commit ID: 884cba8
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-link-check
  • Commit ID: 884cba8
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-code-formatting
  • Commit ID: 884cba8
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@A-yush
uploaded modified spark-processing-sm-core.ipynb
cd25c80 
removed outputs, used default bucket and added additional comments
@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-amazon-sagemaker-examples-pr
  • Commit ID: cd25c80
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-link-check
  • Commit ID: cd25c80
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: SageMakerNotebooksProd-USWEST2-sagemaker-examples-code-formatting
  • Commit ID: cd25c80
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pintaoz-aws pintaoz-aws pintaoz-aws left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@A-yush @sagemaker-bot @pintaoz-aws