End to end tests for important use cases

[chrispomeroyhale]

Is your feature request related to a problem? Please describe.
My team is exploring integrating with the Crowdin SDK which we are excited about. I have been tasked with finding out what happens if we depend on Over-the-Air translations and there's a network failure.

I started writing some additional tests. It became clear that although some aspects of this project are easily testable, the changes required to make LocalizationProvider and all of its dependencies testable is a big lift and I am unfamiliar with this code base.

Describe the solution you'd like
It would be great to see some expansion of test cases to cover business critical functions to give us more confidence about the SDK. For instance, in at least one place a callback is missing -- does this have any real world consequence? Are there any other cases like this?

Some of the SDK is already using protocols which is a great! But some parts are using global state which makes it hard to test.

LocalizationProvider can already be passed in a CrowdinRemoteLocalizationStorage but we are currently unable to pass in a CrowdinLocalizationDownloader in order to use the existing URLSessionMock to simulate successful and failed network respones. This may also require creating a mock for CrowdinContentDeliveryAPI to pass in a manifest. Additionally, although CrowdinContentDeliveryAPI has some test coverage, it does not cover the downloading of the manifest.

Describe alternatives you've considered
I successfully added a mock for RemoteLocalizationStorageProtocol and passed that into CrowdinSDK.startWithRemoteStorage(). However, this doesn't give deep enough test coverage into what happens during a network failure.

Additional context
I may have some time next week to work with you towards a solution.

[Rommka4]

Hi @chrispomeroyhale !
Thank you for sharing all the details with us, our developers will be able to check everything in detail and answer you from Monday
In the meantime, I wish you a good day!🙂

[chrispomeroyhale]

I'd like to share what I have figured out so far, which the developers are free to use or modify or reject.

My first attempt tries to see what happens if the RemoteLocalizationStorageProtocol doesn't return any strings -- simulating what might happen if the network requests fail. The code below shows that the Localization falls back to the translation from the Localizable.strings bundled with the test target.

The pitfall is we aren't actually testing whether the CrowdinRemoteLocalizationStorage functions correctly in various use cases. I found that testing CrowdinRemoteLocalizationStorage is challenging because the code isn't currently set up to pass URLSessionMock, and there is some global state with the Manifest that makes this complicated.

    func testRemoteLocalizationDoesntExist() {
        let started = expectation(description: "SDK started")
        let remoteStorage = RemoteLocalizationStorageMock()
        CrowdinSDK.startWithRemoteStorage(remoteStorage) {
            started.fulfill()
        }
        wait(for: [started], timeout: 60)
        XCTAssertEqual(CrowdinSDK.inBundleLocalizations.count, 3)
        XCTAssertEqual(Localization.current.localizedString(for: "test_key"), "test_value [B]")
        CrowdinSDK.deintegrate()
        CrowdinSDK.stop()
    }

    func testRemoteLocalizationExists() {
        let started = expectation(description: "SDK started")
        let remoteStorage = RemoteLocalizationStorageMock(strings: ["test_key": "test_value [testRemoteLocalizationExists]"])
        CrowdinSDK.startWithRemoteStorage(remoteStorage) {
            started.fulfill()
        }
        wait(for: [started], timeout: 60)
        XCTAssertEqual(CrowdinSDK.inBundleLocalizations.count, 3)
        XCTAssertEqual(Localization.current.localizedString(for: "test_key"), "test_value [testRemoteLocalizationExists]")
        CrowdinSDK.deintegrate()
        CrowdinSDK.stop()
    }

class RemoteLocalizationStorageMock: RemoteLocalizationStorageProtocol {

    let strings: [String: String]
    let plurals: [AnyHashable: Any]

    init(name: String = "Mock", strings: [String: String] = [:], plurals: [AnyHashable: Any] = [:]) {
        self.name = name
        self.strings = strings
        self.plurals = plurals
    }

    // MARK: RemoteLocalizationStorageProtocol

    var name: String

    func prepare(with completion: @escaping () -> Void) {
        completion()
    }

    var localizations: [String] = ["en"]

    var localization: String = "en"

    func deintegrate() {}

    func fetchData(completion: @escaping LocalizationStorageCompletion, errorHandler: LocalizationStorageError?) {
        completion(localizations, localization, strings, plurals)
    }

}

[tattiza]

Hi!
Thanks for the additional comment, will share it with the team
Once we have any news - we'll come back with the reply!