Future of MailTrackerBlocker – Porting from Plugin to Mail Extension
Tekl opened this issue · 42 comments
It seems, that macOS 14 Sonoma will remove the plugin feature in Apple Mail. As Apple said with macOS 11 Big Sur, the only way to extend Mail will be the limited Mail Extensions. In my understanding MailTrackerBlocker could be ported to Extensions. Have you already investigated this case?
Yep, definitely possible, looked into it a few years back. Hadn't considered porting it though because Mail Privacy Protection is a more robust solution
I've read several times that Mail Privacy Protection is supposed to be better. Maybe I'm misunderstanding how it works, but I see it as complimentary to MTB. Let's use an example, here's the latest tracker MTB has blocked:
<img src="https://fromtheheart.us18.list-manage.com/track/open.php?u=3D818a719e5f2462d35cb6f299d&id=3D2dc587eecf&e=3Dd514c7520d" height="1" width="1" alt="">
According to their documentation, Apple's MPP implementation can do 3 things:
-
It can hide my IP address from the web server serving this
<img>by using proxies. But the email marketing platforms don't need to use my IP address to track this, they use the unique identifieru=3D818a719e5f2462d35cb6f299d&id=3D2dc587eecf&e=3Dd514c7520d". That is, tracking occurs at the application layer, not the network layer (I used to do this myself in a previous job; I could see exactly who opened my messages, and it is at the application layer that this tracking was enabled). -
Additionally, it can download content in the background rather than when the message is opened. Apple's documentation claims:
This option prevents senders from collecting information when you view a message, such as when and how many times you view it, whether you forward it, your IP address and other data.
To an extent, this is all correct, but (1) still applies. As the application layer contains a unique identifier which the email marketing platform associates with your email address (plus anything else they know about you), it can (and does) still track you.
- Finally, Apple's MPP implementation can will block all remote content, and this indeed will prevent tracking; until I press the "Load Remote Content" button! Then I've overridden this setting for this instance, and we're back to either being tracked or relying on something like MTB to block tracking at the application layer.
If I've misunderstood anything above, I would be very grateful if someone could set me straight, because it's been bothering me that MPP is being touted as some kind of privacy silver bullet, when its effectiveness seems easily circumvented to me.
tracking occurs at the application layer, not the network layer
Yes, that's right. But MPP also works at the application-level, as a transparent HTTP proxy built into macOS (the NetworkServiceProxy daemon) that works with Mail, similar to what iCloud Private Relay's is for Safari. For a given image, this MPP takes the whole HTTP URL, caches it on Apple's server, then it comes time for Mail to lookup that exact URL when you open that email, the NetworkServiceProxy daemon will transparently redirect the cached content back to Mail instead of going direct to the marketer's server. When the proxy/relay caches the spy pixel, it will trigger it, and yes, the spy pixel will report that it's been opened but I think dhh stated MPP's impact the best when he said::
Either way, and given Apple's monopoly advantage with their preinstalled Mail app, we don't need much of an uptake from what they're calling Mail Privacy Protection to break the dam on spy pixels. You can't really say anything authoritatively about open rates if 5-10-30-50% of your recipients are protected against snooping, as you won't know whether that's why your spy pixel isn't tripping, or it's because they're just not opening your email.
Anyway, for all this to work, you need to check "Protect Mail Activity" instead of just the "Hide IP Address" box. "Load Remote Content" will indeed connect you directly to the marketer's servers and let them know when exactly you opened the mail.
The emailer can't tell if you've opened and viewed the email (because essentially all emails are opened and viewed upon receipt), and they can't tell your IP address, but what they can tell is if your email address is valid and not blocked by a spam filter.
I have MTB off and Mail Privacy Protection turn on on my Monterey machine, but for this reason I'm still not convinced it is better in all respects than MTB.
Thank you both for explaining in more detail how MPP works. Maybe I need to reassess. I appreciate your candour.
Decided to work on the Mail Extension. There's some limitations with the Content Blocker format that make it less effective than the plugin, the biggest one being that I can't setup a generic regex rule for 1x1px img tags anymore. I think it's still possible to capture the majority of trackers like we've been doing with URL targeted rulesets.
Anyhow, I've been going through the App Store review process. It'll still need a lot of work to get it up to same effectiveness as the plugin but let's see how it goes.
I'm using MailTrackerBlocker, because Mail Privacy Protection (and Private Relay which it uses) doesn't work together with Adguard: https://adguard.com/kb/adguard-for-mac/solving-problems/protect-mail-activity/
And so I'd love to see MTB converted to an extension.
A question though: "I can't setup a generic regex rule for 1x1px img tags anymore." - does it mean tracking pixels will be able get through?
A question though: "I can't setup a generic regex rule for 1x1px img tags anymore." - does it mean tracking pixels will be able get through?
If I'm limited to the Safari-style Content Blocker API, some will get through yes.But I've been able to get it to work the same way as the plugin though Mail Kit's message security APIs. Have to wait a bit though; App Review is taking some time.
I did a soft launch of the Mail Extension. App Store-approved and all.
There's unfortunately two major issues that some users have been facing: Mail crashing and/or slow-loading of messages. Because of the limited MailKit API, I'm not able to fix or even workaround these issues. These issues most likely are on the API/Mail side because they seem to occur when I had these users test out Apple sample Mail Extension app. Hoping there will be fixes before Sonoma is out in the wild.
Is this the extension you've softlaunched?
https://apps.apple.com/de/app/mailtrackerblocker/id6450760473
Nice to see the extension ported to Apple Mail's modern add-on type.
The bugs I mentioned above seem to have been resolved in recent Sonoma betas 🚀. So this extension will be officially launched on the same day as the Sonoma 14 release as it will require Sonoma. Ventura 13 and older users should continue to use the existing plugin.
EDIT: still crashes on Sonoma 14.0
So, currently there's no way to get the Extension as you removed it from that App Store?
Not at the moment. I was getting numerous complaints about the above issues with Ventura so I decided pull it until Apple's fixes land (in the form of Sonoma).
Not at the moment. I was getting numerous complaints about the above issues with Ventura so I decided pull it until Apple's fixes land (in the form of Sonoma).
Does this mean that when it becomes live again, those who paid won’t have to repay?
I don’t mind paying for it again myself, but I worry that if it appears to be a new, separate purchase from the original which others already paid for, that it may cause some confusion.
Does this mean that when it becomes live again, those who paid won’t have to repay?
Definitely. If you already have it, you can update to the upcoming 1.0.5 version once it's live but you'll need to upgrade to Sonoma first to use it. If you'd like to stay on Ventura, I recommend staying on the existing plugin.
Could you release the mail extension (app) in the German Mac App Store?
Thanks.
+1 for worldwide access.
Unfortunately through further testing, Mail still crashes on the official Sonoma 14.0 release when my Extension is enabled. Seems to be random. The good news is that Apple has recently acknowledged my issue with the MailKit API but we'll have to wait for a fix from their end still.
As an interim workaround, I've been toying with the idea of releasing the Extension as a Content Blocker-only Extension. It will block trackers but won't be any tracker labeling/reporting functionality which means it won't give you any indication that something has been blocked (due to the Content Blocker API limitation). I'm thinking it's not worth putting it up in this form so I'm still leaning toward waiting for an official fix.
@apparition47 I remember you mentioning something about the new API's limitations compared to the older one, is it stuff like displaying information, or the actual tracker blocking capabilities (compared to the old one)?
I've managed to replicate the original plugin's functionalities with the new APIs (labeling trackers, displaying that info inline in the email viewer pane, blocking trackers not on the blacklist) but one of these APIs causes Mail to crash in its current state. Official APIs definitely shouldn't crash Mail.
The Content Blocker API doesn't crash but doesn't allow me to implement any of the above functions.
It's gonna be great once Apple fixes their end, thank you for your hard work 👍
Hello! Just thought I would check in on the Sonoma support and if this extension was working. Looking forward in installing as I just came across this.
Still waiting for a crucial Mail fix on Apple's side. They've acknowledged my bug back in Sept but haven't seen any update around this as of 14.2 beta 1.
Still waiting for a crucial Mail fix on Apple's side. They've acknowledged my bug back in Sept but haven't seen any update around this as of 14.2 beta 1.
Will it then also be available in other App Stores around the world?
Did Apple still not fix it in 14.1.1?
Will it then also be available in other App Stores around the world?
Yes
Did Apple still not fix it in 14.1.1?
Hasn't been fixed according to https://developer.apple.com/documentation/macos-release-notes/macos-14_1-release-notes
How about 14.2.1?
Apple has since updated my bug report with the status "Potential fix identified - For a future OS update" but there hasn't been anything in 14.2.1 or the 14.3 betas. Will have to continue to wait
Is this your app? https://apps.apple.com/us/app/mailtrackerblocker/id6450760473
@Bob16180 Yes! Apple officially fixed the crash in Sonoma 14.3. There's still a few remaining bugs I need to iron out with Apple but I've put it in the store for now.
Congratulations on getting it released! Any advice for users of the old version to upgrade? Will there be a home-brew cask and/or TestFlight? Or would you recommend buying the App Store version?
@blackketter Thanks! I've already had a few requests for non-App Store distribution. Not sure in what form would be best but looking into this down the line. Please invite yourself to TestFlight so you can try it out.
There's a pretty annoying bug in 1.0.6: apostrophes and quotation marks are not displayed correctly in Mail messages: they appear as "’" and "“", respectively. Accents don't appear correctly, either.
I've filed a bug report.
Sorry for the question, but does this version retain all the blocking capabilities of the original one?
So great to have this back working! A few issues/observations: 1. Mail rendering seems slow now with the extension enabled? 2. There is no indication that something has been blocked. There used to be an icon, now all we see is a generic “plugin” icon which needs to be clicked to see if anything was blocked or not. I am happy to test new versions via TestFlight if required.
…
Message ID: @.*** com>
The extension needs to be enabled. There a puzzle piece/extension icon in the e-mail.
@Bob16180 Yes! Apple officially fixed the crash in Sonoma 14.3. There's still a few remaining bugs I need to iron out with Apple but I've put it in the store for now.
Thank you for continuing to develop this app! It works great.
@Bob16180 - Yes I had it enabled. ;)
Previous behaviour was there was an icon in the top right corner of the email display that showed was solid blue if trackers were blocked. eg the image on the from the main git page https://github.com/apparition47/MailTrackerBlocker:
The solid blue X icon indicated that a tracker was blocked.
Now this is just the generic plugin icon - and you have to click on that to view if there were trackers or not.
There's a pretty annoying bug in 1.0.6: apostrophes and quotation marks are not displayed correctly in Mail messages: they appear as "’" and "“", respectively. Accents don't appear correctly, either.
@odysseus90210 Thanks for the report. I moved it to the new repo. As mentioned, please send me the affected .eml if possible.
Note to anyone else with issues with the app version, please file it on the app repo instead: https://github.com/apparition47/MailTrackerBlockerApp
Sorry for the question, but does this version retain all the blocking capabilities of the original one?
@lothar-cell Yes, core blocking features that exactly the same. The original plugin is pretty mature at this point in contrast to the new app, which has a few bugs I need to work out as you can see in the comments. You are welcome to try out the app if you're on Sonoma 14.3 by using TestFlight.
- Mail rendering seems slow now with the extension enabled?
@malwcal This one is a known issue that requires Apple to fix. I'm trying to flag this one to Apple. If you could, please also file a bug via Feedback Assistant referencing FB12796974.
- There is no indication that something has been blocked. There used
to be an icon, now all we see is a generic “plugin” icon which needs to be
clicked to see if anything was blocked or not.
Apple's new MailKit APIs have limitations like this. I can't change that icon (I'm asking Apple if they can let me change it in FB13518933). In the meanwhile I do have an inline banner version that you could test out. Please invite yourself to the TestFlight to try out version 1.0.5 then let me know if you prefer this.
Will the sources for the new version be available or did you decide to make it closed-source?