Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

localhost mode is not working in testing #34

Closed
ashoksudani opened this issue Nov 5, 2020 · 14 comments
Closed

localhost mode is not working in testing #34

ashoksudani opened this issue Nov 5, 2020 · 14 comments

Comments

@ashoksudani
Copy link

ashoksudani commented Nov 5, 2020

Hi,

We are using react-client in our app, while testing we used localhost mode as mentioned here https://help.split.io/hc/en-us/articles/360020448791-JavaScript-SDK#localhost-mode, but its always returning isReady property to false.

This is the splitprops we are getting, :

splitProps {
        factory: {
          client: [Function: client],
          manager: [Function: manager],
          Logger: {
            enable: [Function: enable],
            setLogLevel: [Function: setLogLevel],
            disable: [Function: disable],
            LogLevel: [Object]
          },
          settings: {
            mode: 'localhost',
            core: [Object],
            scheduler: [Object],
            urls: [Object],
            storage: [Object],
            version: 'react-1.2.1',
            streamingEnabled: true,
            sync: [Object],
            startup: [Object],
            features: [Object],
            runtime: [Object],
            integrations: []
          },
          sharedClientInstances: Set {},
          config: { core: [Object], features: [Object], scheduler: [Object] }
        },
        client: EventEmitter {
          getTreatment: [Function: getTreatment],
          getTreatmentWithConfig: [Function: getTreatmentWithConfig],
          getTreatments: [Function: getTreatments],
          getTreatmentsWithConfig: [Function: getTreatmentsWithConfig],
          track: [Function: track],
          isBrowserClient: false,
          destroy: [Function: destroy]
        },
        isReady: false,
        isReadyFromCache: false,
        isTimedout: false,
        hasTimedout: false,
        isDestroyed: false,
        lastUpdate: 0,
        treatments: { billing_updates: { treatment: 'control', config: null } }
      }

And config being passed is :

{
        core: {
          authorizationKey: 'localhost',
        },
        features: {
          billing_updates: { treatment: 'visa', config: '{ "color": "blue" }' }, //example of a defined config
        },
      }

Any help regarding how to test react-client ?

Tried mocking http requests which is very difficult though as object being returned is very very complex to mock.

@chillaq
Copy link

chillaq commented Nov 5, 2020

Hi, please check the KB article: https://help.split.io/hc/en-us/articles/360047413111-How-to-use-Jest-to-test-React-SDK-

Let me know if it helps.

Thanks
Bilal

@ashoksudani
Copy link
Author

Thanks @chillaq

But this configuration provides a way through enzyme, we are using react testing library.

@chillaq
Copy link

chillaq commented Nov 11, 2020

Hi, The React SDK only works on browser, it does not support server side, are you testing the code on browser side?

@ashoksudani
Copy link
Author

ashoksudani commented Nov 13, 2020

Thanks @chillaq

It runs the code in JSDOM, https://github.com/testing-library/react-testing-library

@ashoksudani
Copy link
Author

ashoksudani commented Dec 15, 2020

@chillaq

Plain JS client is also not working with Jest.

passing down following config:

 {
      core: { authorizationKey: 'localhost' },
      features: {
        flag-xyz': { treatment: 'on', config: null }
      },
      scheduler: { offlineRefreshRate: 15 }
    }

getting splitClient like:

    splitClient EventEmitter {
      getTreatment: [Function: getTreatment],
      getTreatmentWithConfig: [Function: getTreatmentWithConfig],
      getTreatments: [Function: getTreatments],
      getTreatmentsWithConfig: [Function: getTreatmentsWithConfig],
      track: [Function: track],
      isBrowserClient: false,
      destroy: [Function: destroy]
    }

but its never calling callback function passed to

splitClient.on(splitClient.Event.SDK_READY, callBackFunction);

how to test ready state then ?

@chillaq
Copy link

chillaq commented Jan 14, 2021

Hi @ashoksudani , I have a suggestion from Engineering:

Have you tried using craco? https://www.npmjs.com/package/@craco/craco#jest-api

You might be able to use craco and configure that field without ejecting.

Thanks
Bilal

@EmilianoSanchez
Copy link
Contributor

hi @ashoksudani

To let you know, we have released v1.2.4 of the React SDK, which includes an update to solve the issues you were facing when using Jest for unit testing.

Now it is not required to configure Jest with neither the browser option or browser-resolve. In other words, you can now use the default Jest configuration, provided, for example, by create-react-app projects. No need to eject the project or use craco.

Please, let us know if you need further details.

@adrienrn
Copy link

I've been seeing the same thing using v1.2.4.

isReady remains false and in debug we see this:

  console.log
    [INFO]  splitio-client => Split: feature1. Key: localhost_key. Evaluation: control. Label: not ready
...

What I'm seeing is that the SplitTreatments way of doing it is working the hook version (useTreatments and useContext(SplitContext)) is never resolving with isReady always being false.

Here's a reproduction:
https://codesandbox.io/s/cranky-liskov-bg8z0?file=/src/App.test.js

Took me many hours to narrow it down to... hooks, or at least that's as far as I could go. Note that test4 is also using hook but asking for another feature feature2, and it works.

In my apps, I have a useFeature hook (fairly small, calling useTreatments) and I have been struggling so much unit testing it. This could explain why. At first I was thinking I was doing something wrong with the initialization, seeing [WARN] Factory instantiation: You already have 1 factory with this API Key... but that seems to be normal in unit tests environment.


  console.log
    [DEBUG] splitio-producer:task => Starting ObjectUpdater refreshing each 15000

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:task => Running ObjectUpdater

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:offline => Splits data:

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:offline => {"feature1":{"conditions":[{"conditionType":"ROLLOUT","matcherGroup":{"combiner":"AND","matchers":[{"keySelector":null,"matcherType":"ALL_KEYS","negate":false}]},"partitions":[{"treatment":"off","size":100}],"label":"default rule"}],"configurations":{}}}

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [INFO]  splitio => New Split SDK instance created.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio => Retrieving default SDK client.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [WARN]  No listeners for SDK Readiness detected. Incorrect control treatments could have been logged if you called getTreatment/s while the SDK was not yet ready.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-engine:sanitize => Attempted to sanitize [[object Object]] which should be of type [STRING]. 
     Sanitized and processed value => [localhost_key]

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.map (<anonymous>)

  console.log
    [DEBUG] splitio-engine:matcher => [allMatcher] is always true

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.map (<anonymous>)

  console.log
    [DEBUG] splitio-engine:combiner => [andCombiner] evaluates to true

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.map (<anonymous>)

  console.log
    [DEBUG] splitio-engine => [engine] using algo legacy bucket 38 for key localhost_key using seed undefined - treatment off

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.map (<anonymous>)

  console.log
    [DEBUG] splitio-engine:combiner => Treatment found: off

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [INFO]  splitio-client => Split: feature1. Key: localhost_key. Evaluation: off. Label: default rule

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.forEach (<anonymous>)

  console.log
    [INFO]  splitio-client => Queueing corresponding impression.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.forEach (<anonymous>)

  console.log
    [TIME TRACKER] => [SDK - Get Treatments with config] took 6ms to finish.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio => Retrieving default SDK client.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:task => Stopping ObjectUpdater

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [WARN]  Factory instantiation: You already have 1 factory with this API Key. We recommend keeping only one instance of the factory at all times (Singleton pattern) and reusing it throughout your application.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:task => Starting ObjectUpdater refreshing each 15000

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:task => Running ObjectUpdater

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:offline => Splits data:

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:offline => {"feature1":{"conditions":[{"conditionType":"ROLLOUT","matcherGroup":{"combiner":"AND","matchers":[{"keySelector":null,"matcherType":"ALL_KEYS","negate":false}]},"partitions":[{"treatment":"on","size":100}],"label":"default rule"}],"configurations":{}}}

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [INFO]  splitio => New Split SDK instance created.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio => Retrieving default SDK client.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [WARN]  No listeners for SDK Readiness detected. Incorrect control treatments could have been logged if you called getTreatment/s while the SDK was not yet ready.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-engine:sanitize => Attempted to sanitize [[object Object]] which should be of type [STRING]. 
     Sanitized and processed value => [localhost_key]

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.map (<anonymous>)

  console.log
    [DEBUG] splitio-engine:matcher => [allMatcher] is always true

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.map (<anonymous>)

  console.log
    [DEBUG] splitio-engine:combiner => [andCombiner] evaluates to true

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.map (<anonymous>)

  console.log
    [DEBUG] splitio-engine => [engine] using algo legacy bucket 38 for key localhost_key using seed undefined - treatment on

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.map (<anonymous>)

  console.log
    [DEBUG] splitio-engine:combiner => Treatment found: on

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [INFO]  splitio-client => Split: feature1. Key: localhost_key. Evaluation: on. Label: default rule

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.forEach (<anonymous>)

  console.log
    [INFO]  splitio-client => Queueing corresponding impression.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.forEach (<anonymous>)

  console.log
    [TIME TRACKER] => [SDK - Get Treatments with config] took 3ms to finish.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio => Retrieving default SDK client.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:task => Stopping ObjectUpdater

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [WARN]  Factory instantiation: You already have 2 factories with this API Key. We recommend keeping only one instance of the factory at all times (Singleton pattern) and reusing it throughout your application.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:task => Starting ObjectUpdater refreshing each 15000

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:task => Running ObjectUpdater

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [INFO]  splitio => New Split SDK instance created.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio => Retrieving default SDK client.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [WARN]  getTreatmentsWithConfig: the SDK is not ready, results may be incorrect. Make sure to wait for SDK readiness before using this method.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [INFO]  splitio-client => Split: feature1. Key: localhost_key. Evaluation: control. Label: not ready

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.forEach (<anonymous>)

  console.log
    [INFO]  splitio-client => Queueing corresponding impression.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)
          at Array.forEach (<anonymous>)

  console.log
    [TIME TRACKER] => [SDK - Get Treatments with config] took 1ms to finish.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

 FAIL  components/SplitAwareExperimentContext/Sometest.spec.js
  ✓ test1 (58 ms)
  ✓ test2 (14 ms)
  ✕ test3 (1018 ms)

  ● test3

    TestingLibraryElementError: Unable to find an element with the text: READY. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.

    <body>
      <div>
        <span>
          NOT_READY
        </span>
        <span>
          LOADING...
        </span>
      </div>
    </body>

    <html>
      <head />
      <body>
        <div>
          <span>
            NOT_READY
          </span>
          <span>
            LOADING...
          </span>
        </div>
      </body>
    </html>Error: Unable to find an element with the text: READY. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.

    <body>
      <div>
        <span>
          NOT_READY
        </span>
        <span>
          LOADING...
        </span>
      </div>
    </body>

      122 | 
      123 |   // Wait for the SDK to grab the proper treatment.
    > 124 |   await waitFor(() => expect(screen.getByText('READY')).toBeDefined());
          |         ^
      125 |   await waitFor(() => expect(screen.getByText('THE BEES ARE HAPPY')).toBeDefined());
      126 | });
      127 | 

      at waitForWrapper (node_modules/@testing-library/dom/dist/wait-for.js:173:27)
      at Object.<anonymous> (components/SplitAwareExperimentContext/Sometest.spec.js:124:9)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 2 passed, 3 total
Snapshots:   0 total
Time:        4.2 s
Ran all test suites matching /components\/SplitAwareExperimentContext\/Sometest.spec.js/i.
  console.log
    [DEBUG] splitio => Retrieving default SDK client.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

  console.log
    [DEBUG] splitio-producer:task => Stopping ObjectUpdater

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:70:15)

error Command failed with exit code 1.

@ashoksudani
Copy link
Author

ashoksudani commented Sep 1, 2021

it seems fixed in version 1.2.6 Please check #13

But not fully.

When using storage type = 'localstorage' its giving isReadyFromCache to false

factory: {
        client: [Function: client],
        manager: [Function: manager],
        Logger: {
          enable: [Function: enable],
          setLogLevel: [Function: setLogLevel],
          disable: [Function: disable],
          LogLevel: [Object]
        },
        settings: {
          mode: 'localhost',
          core: [Object],
          scheduler: [Object],
          urls: [Object],
          storage: [Object],
          version: 'react-1.2.6',
          streamingEnabled: true,
          sync: [Object],
          startup: [Object],
          debug: true,
          runtime: [Object],
          integrations: []
        },
        sharedClientInstances: Set(0) {},
        config: {
          core: [Object],
          sync: [Object],
          storage: [Object],
          debug: true,
          version: 'react-1.2.6'
        }
      },
      client: EventEmitter {
        getTreatment: [Function: bound getTreatment],
        getTreatmentWithConfig: [Function: bound getTreatmentWithConfig],
        getTreatments: [Function: bound getTreatments],
        getTreatmentsWithConfig: [Function: bound getTreatmentsWithConfig],
        track: [Function: bound track],
        isBrowserClient: true,
        destroy: [Function: destroy],
        _eventsCount: 6
      },
      isReady: true,
      isReadyFromCache: false,
      isTimedout: false,
      hasTimedout: false,
      isDestroyed: false,
      lastUpdate: 1630473144882,
      treatments: { 'alias-linkid-ux': { treatment: 'control', config: null } }
    }
This is my splitconfig
{
  core: {
    authorizationKey: 'localhost',
    key: uid,
    trafficType: 'user',
  },
  features: {
    'alias-linkid-ux': { treatment: 'on', config: null },
  },
  storage: {
    type: 'LOCALSTORAGE',
  },
  debug: process.env.SPLIT_ENV !== 'production',
}
This is really frustrating ! 

@EmilianoSanchez
Copy link
Contributor

EmilianoSanchez commented Sep 16, 2021

Hi @ashoksudani ,

We have considered your issue and we will address it in a coming release, probably for the end of next week.

The current implementation of both React and JS SDKs overrides the 'storage' config param to the default 'MEMORY' type when using localhost mode. That is the reason why isReadyFromCache never gets true in localhost mode. Since next release, the SDK will not override the storage config, and thus you will be able to use LocalStorage in localhost mode.

@EmilianoSanchez
Copy link
Contributor

Hi @ashoksudani . Reaching you back.

We have released JS SDK v10.16.0, and React SDK v1.3.0. Both are minor releases that let use localhost mode with LocalStorage storage type, in which case the SDK emits the SDK_READY_FROM_CACHE event (see line 30 of code snippet), and sets isReadyFromCache prop to true.

@ashoksudani
Copy link
Author

Thank you @EmilianoSanchez for fixing this issue.

I will look forward to use this utility in our testing code for sure. 👍
It will surely help.

@felipeS
Copy link

felipeS commented Dec 13, 2021

I'm facing the same issue with the following code.

  const { isReady:} = useContext(SplitContext);

...
        <div>
          {!isReady ? (
            <div >
              <Skeleton />
              <Skeleton />
              <Skeleton />
            </div>
          ) : (
            <MyComponent />
          )}
        </div>

Log:

      [INFO]  Split SDK is ready.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:59:13)

    console.log
      [WARN]  No listeners for SDK Readiness detected. Incorrect control treatments could have been logged if you called getTreatment/s while the SDK was not yet ready.

      at Logger._log (node_modules/@splitsoftware/splitio-react/lib/splitio/utils/logger/LoggerFactory.js:59:13)

MyComponent never gets to be rendered during the test (jsdom) because the hook never catches the SDK_READY event.

@EmilianoSanchez
Copy link
Contributor

Hi @felipeS ,

It has been a while since your comment, but I wanted to share that we have included a new doc section in our docs (Localhost mode) that includes unit test examples using different testing libraries, like Enzyme and React Testing Library. Tests show how to address the SDK behaviour regarding isReady being false on the first render and then true asynchronously, when the SDK_READY event is triggered. Tests are also available in the public example.

I will close the ticket for now.
Thank you,
Emiliano

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants