IP override for Google Analytics 4

Question

IP override for Google Analytics 4

Closed this issue 3 years ago · 31 comments

The IP override works fine on Google Analytics but not on GA4.
All hits are shown from the server IP.

Answer 1 · 2021-02-15T16:03:58.000Z

Hello! That’s because Google’s measurement protocol v2 does not yet support &uip URL parameter. However, with some magic applied, location override already works in dataunlocker.com for GA4 as well.

…

On Mon, 15 Feb 2021 at 17:42, Ronen Teva ***@***.***> wrote: The IP override works fine on Google Analytics but not on GA4. All hits are shown from the server IP. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#25>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGCCSA5CYQGCCMAVU4UIJ3S7E6FVANCNFSM4XU4TQKQ> .

Answer 2 · 2021-02-15T16:50:16.000Z

Can't this magic be a open-sourced? Or is that a selling point for the paid version?

Answer 3 · 2021-02-15T18:15:04.000Z

Not a selling point. I feel uncomfortable open-sourcing things which will change just tomorrow (read: which are in alpha) and which will be removed from the API completely at some point. Within DataUnlocker, we just can afford experimenting, as once it changes there will be an immediate fix from our side.

The latest Measurement Protocol spec should very soon transition to beta with uip parameter supported. Please wait and take it as "there is no way to do it before MPv2 goes beta". That's Google's problem that they push GAv2 to their customers before updating the protocol at the end of the day :P

P.S. DataUnlocker isn't the paid version of this open sourced app/container; it uses a principally different and a better approach to proxy "everything", without any common code.

Answer 4 · 2021-02-16T07:27:52.000Z

Got it, that makes sense.
Thanks for explaining!

Answer 5 · 2021-06-19T10:26:18.000Z

Regarding this issue, I understand your comments, @ZitRos, how https://developers.google.com/tag-manager/serverside manage to accomplish this task then?

Answer 6 · 2021-06-19T10:52:33.000Z

In fact, I've never tried Google's server-side tagging solution. From here I can see that it doesn't solve the ad blocking problem (replacing the domain isn't enough).

To answer your question, @julesGoullee, did you try server-side GTM with GA4? Does it work with locations?

Answer 7 · 2021-06-19T10:58:42.000Z

I didn't try it too. I thought the purpose of this is to bypass adblocker following the explanation schema. Do you have any private info about when the new GA4 will support uip params ? Seems this issue is open for months, I guess the GA4 homepage show alpha for months as well

Answer 8 · 2021-06-19T11:16:27.000Z

I can't share the release dates nor any other related info (common sense).

However to answer the original question of this issue, I think it is no secret now that Google seems to prefix alpha/beta URL parameters with underscore _, so does &_uip=x.x.x.x work well for Google Analytics 4. I am pretty sure server-side GTM uses it as well if it works with GA4.

I would hold from updating the code of this open-sourced container with _uip, however, given that the new protocol release already takes months we can do it I think. Does it sound good?

Answer 9 · 2021-06-19T11:33:30.000Z

Wow. Didn't know it before, was a secret at least for me! I've tried it out, works perfectly! I can make a pr for this repo, definitely not a big deal to prefix the parameter, I guess everyone will appreciate having it right now when it goes officially released reverting these changes should be smooth IMO.
Thx for your help

Answer 10 · 2021-06-19T11:35:42.000Z

Yes please go ahead and contribute! I wish I have added some very basic tests to this repo when I started it (I test it manually) but this should be a very straightforward fix.

Answer 11 · 2021-06-19T11:41:49.000Z

Let's go! #28

Answer 12 · 2021-06-19T13:03:17.000Z

This is now updated in docker hub zitros/analytics-saviour. @julesGoullee are you using that container? Please confirm it works, I didn't run it. Thanks for contributing!

Answer 13 · 2021-06-19T13:21:48.000Z

I started with the container, work perfectly then I faced issues, I went to deployment from source, I think now I will consider moving back to the container bundle

Answer 14 · 2021-06-19T13:59:06.000Z

Let me know how it goes - I'll keep the issue open for now.

Answer 15 · 2021-06-28T21:42:00.000Z

I assume it's verified now. Thanks!

Answer 16 · 2021-06-29T04:51:32.000Z

The changes work well, the container I didn't test it yet. I will contact you later if I have any issue with the container

Answer 17 · 2021-06-29T08:14:21.000Z

Thanks!

…

On Tue, 29 Jun 2021 at 07:51, jules goullée ***@***.***> wrote: The changes work well, the container I didn't test it yet. I will contact you later if I have any issue with the container — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGCCSHN4RYFCEJBHQAFPITTVFGN3ANCNFSM4XU4TQKQ> .

Answer 18 · 2021-10-26T23:55:11.000Z

Were there any more changes required for this for GA4 property? It looks like we're only seeing the IP of the proxy than the user.

Answer 19 · 2021-10-27T09:04:30.000Z

@iammattmartin double check whether you're passing X-Forwarded-For and X-Real-Ip headers to the back end if you're using something like nginx in between.

Answer 20 · 2021-10-27T16:34:19.000Z

I believe it was just GA and their delayed reporting. We can see some data coming through from near 24 hours ago now.

Answer 21 · 2022-06-30T18:22:28.000Z

Can someone confirm if _uip is still working?

Answer 22 · 2022-07-04T04:34:38.000Z

Hi @wilsonpage! You should be able to confirm it easily by making a hit. Then, in GA realtime panel you'll see the location corresponding to that IP.

Answer 23 · 2022-07-06T10:48:44.000Z

Hi @wilsonpage! You should be able to confirm it easily by making a hit. Then, in GA realtime panel you'll see the location corresponding to that IP.

Ah, it's working for me now! I was trying to use the official GA4 Measurement Protocol, but the API seems half-baked and doesn't have all the options the unofficial/private API (used in gtag.js) has.

Answer 24 · 2022-07-06T10:49:40.000Z

Could you document it?

Answer 25 · 2022-07-06T10:51:31.000Z

@iammattmartin this is my test script I was using:

/* eslint-disable no-console */
import fetch from 'node-fetch';

(async () => {
  const url = new URL('https://google-analytics.com/g/collect?v=2');

  url.searchParams.set('tid', 'G-BLVBF7CC76');
  // url.searchParams.set('gtm', '2oe6t0');
  url.searchParams.set('_z', 'ccd.v9B');

  // client id
  url.searchParams.set('cid', '1656610788');

  // session id
  url.searchParams.set('sid', '1656610180');

  // user language
  url.searchParams.set('ul', 'en-gb');

  // screen resolution
  url.searchParams.set('sr', '1440x900');

  //
  url.searchParams.set('_s', '1');

  // Count of sessions recorded by GA4. This value increases by one each time a new session is detected ( when the session expires )
  url.searchParams.set('sct', '1');

  // session engagement
  url.searchParams.set('seg', '1');

  // document location
  url.searchParams.set('dl', 'https://example.com/3');

  // document referrer
  url.searchParams.set('dr', 'https://google.com/');

  // document title
  url.searchParams.set('dt', 'Example title');

  // event name
  url.searchParams.set('en', 'page_view');

  // ?
  url.searchParams.set('_nsi', '1');

  // user id (determines location)
  url.searchParams.set('uip', '31.29.124.235');
  url.searchParams.set('_uip', '31.29.124.235');

  // event parameters
  url.searchParams.set('epn.artistId', '123');

  // ?
  url.searchParams.set('_ee', '1');

  // first_view
  // url.searchParams.set('_fv', '1');

  // session_start
  // url.searchParams.set('_ss', '1');

  console.log(url);

  const res = await fetch(url.toString(), {
    method: 'POST',
    headers: {
      Accept: '*/*',
      'Cache-Control': 'no-cache',
      Pragma: 'no-cache',
      'Sec-Ch-Ua-Mobile': '?0',
      'Sec-Fetch-Dest': 'empty',
      'Sec-Fetch-Mode': 'no-cors',
      'Sec-Fetch-Site': 'cross-site',
      'User-Agent':
        'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
    },
  });

  console.log(res.status);
})();

Answer 26 · 2022-07-06T10:52:06.000Z

This cheatsheet is also handy:
https://www.thyngster.com/ga4-measurement-protocol-cheatsheet/

Answer 27 · 2022-07-06T10:53:36.000Z

Thanks but in terms of integration within this specific project. Did you make changes to the .js? Would be good if you could submit or fork it so people can use :)

Answer 28 · 2022-07-06T10:55:35.000Z

@iammattmartin I didn't use this project directly, I have my own proxy event service that consumes abstract events and then forwards them out to other services (like GA4). I have a GA3 implementation already, but have been working on the GA4 replacement.

Answer 29 · 2022-07-06T10:56:11.000Z

Ah ok, thanks.

Answer 30 · 2023-11-03T13:46:32.000Z

I'm making requests to www.google-analytics.com/mp/collect with the following parameters:

Query:

_uip
uip
ua

Header:

X-Forwarded-For
User-Agent

Body:

user_id (unique for each user, custom value)
client_id

Despite these details, Google Analytics is only recording hits from my server as a single user.
I made sure that all IP/User-Ageent values is the IP/User-Agent of the user.
I'm using the same value for client_id as I am for user_id.

Is there a step or setting I'm overlooking?

Answer 31 · 2023-11-04T09:29:10.000Z

@ThiloReintjes IP overrides work at least for /g/collect endpoint (it follows almost the same spec as /mp/collect). I doubt that /mp/collect specifically supports or will ever support IP overrides, as even this issue is P4.