Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow specifying "Cold" tier #2096

Merged
merged 3 commits into from
Mar 21, 2023
Merged

Allow specifying "Cold" tier #2096

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f99b0bc
Allow specifying "Cold" tier
adreed-msft Mar 1, 2023
562e86f
Switch service version when setting tier
adreed-msft Mar 6, 2023
38f6a20
Merge remote-tracking branch 'origin/dev' into adreed/cold-tier
adreed-msft Mar 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions common/fe-ste-models.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -759,6 +759,7 @@ func (BlockBlobTier) None() BlockBlobTier { return BlockBlobTier(0) }
func (BlockBlobTier) Hot() BlockBlobTier { return BlockBlobTier(1) }
func (BlockBlobTier) Cool() BlockBlobTier { return BlockBlobTier(2) }
func (BlockBlobTier) Archive() BlockBlobTier { return BlockBlobTier(3) }
func (BlockBlobTier) Cold() BlockBlobTier { return BlockBlobTier(4) }

func (bbt BlockBlobTier) String() string {
return enum.StringInt(bbt, reflect.TypeOf(bbt))
Expand Down
13 changes: 13 additions & 0 deletions e2etest/zt_preserve_access_tier_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,19 @@ import (
"testing"
)

//func TestTier_UploadCold(t *testing.T) {
// RunScenarios(t, eOperation.Copy(), eTestFromTo.Other(common.EFromTo.LocalBlob()), eValidate.Auto(), anonymousAuthOnly, anonymousAuthOnly, params{
adreed-msft marked this conversation as resolved.
Show resolved Hide resolved
// recursive: true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add an item in our backlog to remind us to bring this test back once the service is live?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backlog items tend to get lost in the weeds, but yes, we should do this. I think we should keep it in the current release and just push it back another step each time while we wait so it gets brought up in the sync-up, rather than stick it in "el backlog del backlog backlog" for it to get lost to time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feature is available in the public preview regions like central Canada.

// accessTier: common.EBlockBlobTier.Cold().ToAccessTierType(), // this is not valid yet on the service, so this test is disabled.
// }, nil, testFiles{
// defaultSize: "4M",
// shouldTransfer: []interface{}{
// folder(""), // root folder
// f("filea"),
// },
// }, EAccountType.Classic(), EAccountType.Standard(), "")
//}

func TestTier_V2ToClassicAccount(t *testing.T) {

RunScenarios(t, eOperation.Copy(), eTestFromTo.Other(common.EFromTo.BlobBlob()), eValidate.AutoPlusContent(), anonymousAuthOnly, anonymousAuthOnly, params{
Expand Down
11 changes: 11 additions & 0 deletions ste/mgr-JobPartMgr.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,17 @@ func NewBlobPipeline(c azblob.Credential, o azblob.PipelineOptions, r XferRetryO
pipeline.MethodFactoryMarker(), // indicates at what stage in the pipeline the method factory is invoked
// NewPacerPolicyFactory(p),
NewVersionPolicyFactory(),
// Bump the service version when using the Cold access tier.
pipeline.FactoryFunc(func(next pipeline.Policy, po *pipeline.PolicyOptions) pipeline.PolicyFunc {
// TODO: Remove me when bumping the service version is no longer relevant.
return func(ctx context.Context, request pipeline.Request) (pipeline.Response, error) {
if request.Header.Get("x-ms-access-tier") == common.EBlockBlobTier.Cold().String() {
request.Header.Set("x-ms-version", "2021-12-02")
}

return next.Do(ctx, request)
}
}),
NewRequestLogPolicyFactory(RequestLogOptions{
LogWarningIfTryOverThreshold: o.RequestLog.LogWarningIfTryOverThreshold,
SyslogDisabled: common.IsForceLoggingDisabled(),
Expand Down
2 changes: 1 addition & 1 deletion ste/xfer-anyToRemote-file.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ func BlobTierAllowed(destTier azblob.AccessTierType) bool {
// Standard storage account. If it's Hot, Cool, or Archive, we're A-OK.
// Page blobs, however, don't have an access tier on Standard accounts.
// However, this is also OK, because the pageblob sender code prevents us from using a standard access tier type.
return destTier == azblob.AccessTierArchive || destTier == azblob.AccessTierCool || destTier == azblob.AccessTierHot
return destTier == azblob.AccessTierArchive || destTier == azblob.AccessTierCool || destTier == common.EBlockBlobTier.Cold().ToAccessTierType() || destTier == azblob.AccessTierHot
adreed-msft marked this conversation as resolved.
Show resolved Hide resolved
adreed-msft marked this conversation as resolved.
Show resolved Hide resolved
}
}

Expand Down