vapor/queues

"JobsCommand did not shutdown before deinit" when run unit tests

3a4oT opened this issue · 2 comments

3a4oT commented

Steps to reproduce

I use the QUEUES package and try to understand how to use it. I have created a simple Job which fetches data from a remote server. I config vapor application with the following function

// configures your application
public func configure(_ app: Application) throws {
    // uncomment to serve files from /Public folder
    // app.middleware.use(FileMiddleware(publicDirectory: app.directory.publicDirectory))

    app.databases.use(.sqlite(.file("db.sqlite")), as: .sqlite)
    
    try app.queues.use(.redis(url: "redis://127.0.0.1:6379"))

    //Register jobs
    let categoriesFetcherJob = FetcherJob()
    app.queues.add(categoriesFetcherJob)
    applyWPV2Migrations(app)
    
    try app.queues.startInProcessJobs(on: .default)
    // register routes
    try routes(app)
}

I dispatch Job from Controller like this:

    func index(req: Request) throws -> EventLoopFuture<[String: String]> {
        let jobPayload = CategoriesFetcherJobPayload(startPage: 1, categoriesPerPage: 10)
        return req.queue.dispatch(FetcherJob.self, jobPayload)
            .flatMap { (_) -> EventLoopFuture<[String: String]> in
                let res = ["status": "Fetching of remote categories was scheduled successfully"]
                return req.eventLoop.makeSucceededFuture(res)
        }
    }

The issue occurs when I run unit tests not necessarily related to JOB functionality:

    func testAssert() throws {
        let app = Application(.testing)
        defer {  app.shutdown() }
        try configure(app)
        XCTAssertTrue(true)
} 

After execution, I always hit assertion assert(self.didShutdown, "JobsCommand did not shutdown before deinit")

#6 0x0000000105535f34 in QueuesCommand.deinit at /queues/Sources/Queues/QueuesCommand.swift:164

OR

        `assertionFailure("Command handler deinit when queue is not empty! Queue size: \(self.commandResponseQueue.count)")`

#7 0x00000001053f0a07 in RedisCommandHandler.deinit at /RediStack/Sources/RediStack/ChannelHandlers/RedisCommandHandler.swift:43

Expected behavior

Tests do not crash

Actual behavior

Hit assertion related to JOB but there is no way to shutdown it properly.

Environment

 dependencies: [
        // 💧 A server-side Swift web framework.
        .package(url: "https://github.com/vapor/vapor.git", from: "4.14.0"),
        .package(url: "https://github.com/vapor/fluent.git", from: "4.0.0"),
        .package(url: "https://github.com/vapor/fluent-sqlite-driver.git", from: "4.0.0-rc.2"),
        .package(url: "https://github.com/vapor/queues-redis-driver.git", from: "1.0.0-rc.3"),
        // .package(url: "https://github.com/apple/swift-protobuf.git", from: "1.8.0")
    ],

macOS 10.15.4

jdmcd commented

Hi there, thanks for opening this issue! This seems related to #72 which I haven't had time to track down yet. Hopefully sometime next week, unless someone from the community wants to dig into it.

3a4oT commented

Hi @jdmcd , thanks for the quick feedback. I've submitted PR