Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static pipeline queue length number #461

Closed
efkan opened this issue May 12, 2017 · 2 comments
Closed

Static pipeline queue length number #461

efkan opened this issue May 12, 2017 · 2 comments

Comments

@efkan
Copy link

efkan commented May 12, 2017

Hi,

Actually there is a static with named as _queue already.
But using it seems very raw usage:

pipeline._queue.length

The code below might be more sense:

pipeline.stackCount() or pipeline.commandCount()

In fact I prefer _queue.length usage because of it works synchronously. Nevertheless I made this suggestion.

@luin
Copy link
Collaborator

luin commented May 12, 2017

In which case do you need to know the length of a pipeline queue?

@efkan
Copy link
Author

efkan commented May 12, 2017

When using async loop, I have to execute pipeline to clear Node.js stack like the following pseudo code.
(Instead of _queue I've applied another variable [let pipelineCmdCount = 0] to count commands which have been put into the pipeline)

Pseudo code:

let pipeline = redis.pipeline()
let someNum = 10000
let i = 0
let test = () => { return i < someNum }
let repeatIt = (done) => {
    if (someCondition == "user") {
        let key = "user:"+ user_id
        pipeline.set(key, someValue)
        if (pipeline._queue.length % 1000 == 0) { // clearing v8 engine stack to prevent max stack size exceeded error
            pipeline.exec((err, results) => {
                pipeline = redis.pipeline()
                setTimeout(done)
            })
        } else {
            process.next(done)
        }
    } else {
       let class = users[i].classrooms
        pipeline.set(someKey, someValue)
        let j = 0
        let test2 = () => { j < class.students.length }
        let repeatIt2 = (done2) => {
            let student_id = class.students[j]._id
            let key = "student:" + student_id
            pipeline.set(key, someValue)
            if (pipeline._queue.length % 1000 == 0) { // clearing v8 engine stack to prevent max stack size exceeded error
                pipeline.exec((err, results) => {
                    pipeline = redis.pipeline()
                    setTimeout(done2)
                }
            } else {
                process.next(done2)
            }
        }
        let results2 = (err, results) => {
            if (pipeline._queue.length % 1000 != 0) {
                pipeline.exec((err, results) => {
                    pipeline = redis.pipeline()
                    done()
                }
            } else {
                done()
            }
        }
        asyncModule.whilst(test2, repeatIt2, results2)
    }
}
let results = (err, res) => {
    if (pipeline._queue.length % 1000 != 0) {
        pipeline.exec((err, results) => {
            // Completed
        })
    } else {
        // Completed
    }
}
asyncModule.whilst(test, repeatIt, results)

@luin luin closed this as completed in a6060cb May 18, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants