Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove unnecessary brace and semicolon in 'putBlockInfo.synchronize' block #411

Closed
wants to merge 2 commits into from

Conversation

CrazyJvm
Copy link
Contributor

delete semicolon

delete semicolon
@@ -678,7 +678,7 @@ private[spark] class BlockManager(
case ArrayBufferValues(array) =>
tachyonStore.putValues(blockId, array, level, false)
case ByteBufferValues(bytes) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you remove the { and } here also? thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems that the whole code in block 'putBlockInfo.synchronized' are in the same style with brace unnecessary...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you don't mind changing those that'd be great!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all are removed i think ... please review & test

@rxin
Copy link
Contributor

rxin commented Apr 15, 2014

Jenkins, add to whitelist.

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@rxin
Copy link
Contributor

rxin commented Apr 15, 2014

Jenkins, add to whitelist.

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build triggered.

@CrazyJvm CrazyJvm changed the title style fix remove unnecessary brace and semicolon in 'putBlockInfo.synchronize' block Apr 15, 2014
@AmplabJenkins
Copy link

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14137/

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14140/

@rxin
Copy link
Contributor

rxin commented Apr 17, 2014

Thanks. I've merged this

asfgit pushed a commit that referenced this pull request Apr 17, 2014
…block

delete semicolon

Author: Chen Chao <[email protected]>

Closes #411 from CrazyJvm/patch-5 and squashes the following commits:

72333a3 [Chen Chao] remove unnecessary brace
de5d9a7 [Chen Chao] style fix

(cherry picked from commit 016a877)
Signed-off-by: Reynold Xin <[email protected]>
@asfgit asfgit closed this in 016a877 Apr 17, 2014
pwendell added a commit to pwendell/spark that referenced this pull request May 12, 2014
Improved logic of finding new files in FileInputDStream

Earlier, if HDFS has a hiccup and reports a existence of a new file (mod time T sec) at time T + 1 sec, then fileStream could have missed that file. With this change, it should be able to find files that are delayed by up to <batch size> seconds. That is, even if file is reported at T + <batch time> sec, file stream should be able to catch it.

The new logic, at a high level, is as follows. It keeps track of the new files it found in the previous interval and mod time of the oldest of those files (lets call it X). Then in the current interval, it will ignore those files that were seen in the previous interval and those which have mod time older than X. So if a new file gets reported by HDFS that in the current interval, but has mod time in the previous interval, it will be considered. However, if the mod time earlier than the previous interval (that is, earlier than X), they will be ignored. This is the current limitation, and future version would improve this behavior further.

Also reduced line lengths in DStream to <=100 chars.
pdeyhim pushed a commit to pdeyhim/spark-1 that referenced this pull request Jun 25, 2014
…block

delete semicolon

Author: Chen Chao <[email protected]>

Closes apache#411 from CrazyJvm/patch-5 and squashes the following commits:

72333a3 [Chen Chao] remove unnecessary brace
de5d9a7 [Chen Chao] style fix
bzhaoopenstack pushed a commit to bzhaoopenstack/spark that referenced this pull request Sep 11, 2019
- Add OS_USER_DOMAIN_ID and OS_PROJECT_DOMAIN_ID for Mitaka jobs
- Change the AUTH_URL by adding '/v3' postfix for M+ jobs

Closes: theopenlab/openlab#160
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants