Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --disable-admin-operations flag in Compactor UI and Bucket UI #6646

Merged
merged 8 commits into from
Aug 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re
- [#6264](https://github.com/thanos-io/thanos/pull/6264) Query: Add Thanos logo in navbar
- [#6234](https://github.com/thanos-io/thanos/pull/6234) Query: Add ability to switch between `thanos` and `prometheus` engines dynamically via UI and API.
- [#6346](https://github.com/thanos-io/thanos/pull/6346) Query: Add ability to generate SQL-like query explanations when `thanos` engine is used.
- [#6646](https://github.com/thanos-io/thanos/pull/6646) Compact and Bucket: Add `--disable-admin-operations` flag in Compactor UI and Bucket UI

### Fixed
- [#6503](https://github.com/thanos-io/thanos/pull/6503) *: Change the engine behind `ContentPathReloader` to be completely independent of any filesystem concept. This effectively fixes this configuration reload when used with Kubernetes ConfigMaps, Secrets, or other volume mounts.
Expand Down
3 changes: 3 additions & 0 deletions cmd/thanos/compact.go
Original file line number Diff line number Diff line change
Expand Up @@ -678,6 +678,7 @@ type compactConfig struct {
skipBlockWithOutOfOrderChunks bool
progressCalculateInterval time.Duration
filterConf *store.FilterConfig
disableAdminOperations bool
}

func (cc *compactConfig) registerFlag(cmd extkingpin.FlagClause) {
Expand Down Expand Up @@ -786,4 +787,6 @@ func (cc *compactConfig) registerFlag(cmd extkingpin.FlagClause) {
cc.webConf.registerFlag(cmd)

cmd.Flag("bucket-web-label", "External block label to use as group title in the bucket web UI").StringVar(&cc.label)

cmd.Flag("disable-admin-operations", "Disable UI/API admin operations like marking blocks for deletion and no compaction.").Default("false").BoolVar(&cc.disableAdminOperations)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we call it "web.disable-admin-operations" maybe?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was actaully told by @saswatamcode to make it this way. Still I can ofcourse change the flag name, but still how will the PR get merged when these flaky e2e tests don't pass!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could merge without them passing if they are too flaky and make them stable in another PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we call it "web.disable-admin-operations" maybe?

btw calling it web.disable-admin-operations doesn't seem to be right as it also disables use of API. and the checks are passing now!! i can get a merge maybe

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, API & UI both get disabled by the flag, so web prefix is probably not needed. 🙂

but still how will the PR get merged when these flaky e2e tests don't pass!

No need to worry so much, there are a few common flakes, which pass with 1/2 retries. I think there are issues to address them.

}
17 changes: 10 additions & 7 deletions cmd/thanos/tools_bucket.go
Original file line number Diff line number Diff line change
Expand Up @@ -112,13 +112,14 @@ type bucketLsConfig struct {
}

type bucketWebConfig struct {
webRoutePrefix string
webExternalPrefix string
webPrefixHeaderName string
webDisableCORS bool
interval time.Duration
label string
timeout time.Duration
webRoutePrefix string
webExternalPrefix string
webPrefixHeaderName string
webDisableCORS bool
interval time.Duration
label string
timeout time.Duration
disableAdminOperations bool
}

type bucketReplicateConfig struct {
Expand Down Expand Up @@ -203,6 +204,8 @@ func (tbc *bucketWebConfig) registerBucketWebFlag(cmd extkingpin.FlagClause) *bu
cmd.Flag("timeout", "Timeout to download metadata from remote storage").Default("5m").DurationVar(&tbc.timeout)

cmd.Flag("label", "External block label to use as group title").StringVar(&tbc.label)

cmd.Flag("disable-admin-operations", "Disable UI/API admin operations like marking blocks for deletion and no compaction.").Default("false").BoolVar(&tbc.disableAdminOperations)
return tbc
}

Expand Down
3 changes: 3 additions & 0 deletions docs/components/compact.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,9 @@ Flags:
block loaded, or compactor is ignoring the
deletion because it's compacting the block at
the same time.
--disable-admin-operations
Disable UI/API admin operations like marking
blocks for deletion and no compaction.
--downsample.concurrency=1
Number of goroutines to use when downsampling
blocks.
Expand Down
3 changes: 3 additions & 0 deletions docs/components/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,9 @@ usage: thanos tools bucket web [<flags>]
Web interface for remote storage bucket.

Flags:
--disable-admin-operations
Disable UI/API admin operations like marking
blocks for deletion and no compaction.
-h, --help Show context-sensitive help (also try
--help-long and --help-man).
--http-address="0.0.0.0:10902"
Expand Down
22 changes: 14 additions & 8 deletions pkg/api/blocks/v1.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,13 @@ import (

// BlocksAPI is a very simple API used by Thanos Block Viewer.
type BlocksAPI struct {
baseAPI *api.BaseAPI
logger log.Logger
globalBlocksInfo *BlocksInfo
loadedBlocksInfo *BlocksInfo
disableCORS bool
bkt objstore.Bucket
baseAPI *api.BaseAPI
logger log.Logger
globalBlocksInfo *BlocksInfo
loadedBlocksInfo *BlocksInfo
disableCORS bool
bkt objstore.Bucket
disableAdminOperations bool
}

type BlocksInfo struct {
Expand Down Expand Up @@ -61,6 +62,7 @@ func parse(s string) ActionType {

// NewBlocksAPI creates a simple API to be used by Thanos Block Viewer.
func NewBlocksAPI(logger log.Logger, disableCORS bool, label string, flagsMap map[string]string, bkt objstore.Bucket) *BlocksAPI {
disableAdminOperations := flagsMap["disable-admin-operations"] == "true"
return &BlocksAPI{
baseAPI: api.NewBaseAPI(logger, disableCORS, flagsMap),
logger: logger,
Expand All @@ -72,8 +74,9 @@ func NewBlocksAPI(logger log.Logger, disableCORS bool, label string, flagsMap ma
Blocks: []metadata.Meta{},
Label: label,
},
disableCORS: disableCORS,
bkt: bkt,
disableCORS: disableCORS,
bkt: bkt,
disableAdminOperations: disableAdminOperations,
}
}

Expand All @@ -87,6 +90,9 @@ func (bapi *BlocksAPI) Register(r *route.Router, tracer opentracing.Tracer, logg
}

func (bapi *BlocksAPI) markBlock(r *http.Request) (interface{}, []error, *api.ApiError, func()) {
if bapi.disableAdminOperations {
return nil, nil, &api.ApiError{Typ: api.ErrorBadData, Err: errors.New("Admin operations are disabled")}, func() {}
}
idParam := r.FormValue("id")
actionParam := r.FormValue("action")
detailParam := r.FormValue("detail")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ describe('BlockDetails', () => {
selectBlock: (): void => {
// do nothing
},
disableAdminOperations: false,
};
window.URL.createObjectURL = jest.fn();
const blockDetails = mount(<BlockDetails {...defaultProps} />);
Expand Down
47 changes: 26 additions & 21 deletions pkg/ui/react-app/src/thanos/pages/blocks/BlockDetails.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@ import { download } from './helpers';
export interface BlockDetailsProps {
block: Block | undefined;
selectBlock: React.Dispatch<React.SetStateAction<Block | undefined>>;
disableAdminOperations: boolean;
}

export const BlockDetails: FC<BlockDetailsProps> = ({ block, selectBlock }) => {
export const BlockDetails: FC<BlockDetailsProps> = ({ block, selectBlock, disableAdminOperations }) => {
const [modalAction, setModalAction] = useState<string>('');
const [detailValue, setDetailValue] = useState<string | null>(null);

Expand Down Expand Up @@ -100,26 +101,30 @@ export const BlockDetails: FC<BlockDetailsProps> = ({ block, selectBlock }) => {
<Button>Download meta.json</Button>
</a>
</div>
<div style={{ marginTop: '12px' }}>
<Button
onClick={() => {
setModalAction('DELETION');
setDetailValue('');
}}
>
Mark Deletion
</Button>
</div>
<div style={{ marginTop: '12px' }}>
<Button
onClick={() => {
setModalAction('NO_COMPACTION');
setDetailValue('');
}}
>
Mark No Compaction
</Button>
</div>
{!disableAdminOperations && (
<div>
<div style={{ marginTop: '12px' }}>
<Button
onClick={() => {
setModalAction('DELETION');
setDetailValue('');
}}
>
Mark Deletion
</Button>
</div>
<div style={{ marginTop: '12px' }}>
<Button
onClick={() => {
setModalAction('NO_COMPACTION');
setDetailValue('');
}}
>
Mark No Compaction
</Button>
</div>
</div>
)}
<Modal isOpen={!!modalAction}>
<ModalBody>
<ModalHeader toggle={() => setModalAction('')}>
Expand Down
6 changes: 5 additions & 1 deletion pkg/ui/react-app/src/thanos/pages/blocks/Blocks.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import { sortBlocks, getBlockByUlid, getFilteredBlockPools } from './helpers';
import styles from './blocks.module.css';
import TimeRange from './TimeRange';
import Checkbox from '../../../components/Checkbox';
import { FlagMap } from '../../../pages/flags/Flags';

export interface BlockListProps {
blocks: Block[];
Expand Down Expand Up @@ -74,6 +75,9 @@ export const BlocksContent: FC<{ data: BlockListProps }> = ({ data }) => {
const filteredBlocks = useMemo(() => getBlockByUlid(blocks, blockSearch), [blocks, blockSearch]);
const filteredBlockPools = useMemo(() => getFilteredBlockPools(blockPools, filteredBlocks), [filteredBlocks, blockPools]);

const { response: flagsRes } = useFetch<FlagMap>(`/api/v1/status/flags`);
const disableAdminOperations = flagsRes?.data?.['disable-admin-operations'] === 'true' || false;

const setViewTime = (times: number[]): void => {
setQuery({
'min-time': times[0],
Expand Down Expand Up @@ -180,7 +184,7 @@ export const BlocksContent: FC<{ data: BlockListProps }> = ({ data }) => {
onChange={setViewTime}
/>
</div>
<BlockDetails selectBlock={selectBlock} block={selectedBlock} />
<BlockDetails selectBlock={selectBlock} block={selectedBlock} disableAdminOperations={disableAdminOperations} />
</div>
</>
) : (
Expand Down