Improve the indexability (SEO) of Flutter apps on the web
deepak-kumar-swain opened this issue Β· 199 comments
Latest status: #46789 (comment)
I Just wanted to know whether its SEO Friendly or not and about the status of Initial paintful load.
in the near future, I want to make a commerce site in flutter but for the moment without SEO this option seems compromised
in the near future, I want to make a commerce site in flutter but for the moment without SEO this option seems compromised
You can use https://angel-dart.dev/ for that issue.
Dear Flutter Team,
I just want to know whether you have SEO on your priority or not. If yes, what is the timeline?
Flutter is too good a framework to let go just because of this drawback. (That too from Google.)
This will greatly help make decisions.
Sorry for a bit harsh lines. (But Google is the most used search engine!!!!!!)
Thanks,
Vinod
Currently, Flutter use rendering to html, not the <tag>, so the seo is obviously not a near future for web. Furthermore, just like SPA is coming serveral years, but no support by crawler parser, so there should be hopeless If you only count on flutter. I think @virgiltu 's quote is the best solution to the seo, since It use jael-template and backend technique, so the seo is under full-controll.
Dear Flutter Team,
I just want to know whether you have SEO on your priority or not. If yes, what is the timeline?
Flutter is too good a framework to let go just because of this drawback. (That too from Google.)
This will greatly help make decisions.
Sorry for a bit harsh lines. (But Google is the most used search engine!!!!!!)Thanks,
Vinod
If SEO is something you need than you should use something like AMP on the front end. This cant be done by Flutter. When SEO is a priority there are a lot of other things you should consider. Like Image optimization, website load time, structure, schema and so on. Honestly if you make a website in Flutter you should only use that for the admin side or maybe for the portion of the site that is only seen by the user that is already logged in. If search engines have access to it, you would not want to use Flutter for web, Angular or what ever. Please look at the AMP project. The best example for a flutter web store would be Wish.com or Letgo.com They do not rely on SEO. A web app is not the same as website. Hope this makes sense.
For me That doesnβt make Sense. I love Flutter as a Developer. Iβm working as a Consultant building softwareplatforms for big customers. I hoped that we can use Flutter as the one frontend technologie in Future.so we can Build Widget libraries Based on the Corporate Identity and use them on any plattform.
It would be sad if there is no, somehow, approach for the seo. This would be a main Requirement and a exclusion criterion.
SEO on other frontend frameworks it's not great anyway..
For me That doesnβt make Sense. I love Flutter as a Developer. Iβm working as a Consultant building softwareplatforms for big customers. I hoped that we can use Flutter as the one frontend technologie in Future.so we can Build Widget libraries Based on the Corporate Identity and use them on any plattform.
It would be sad if there is no, somehow, approach for the seo. This would be a main Requirement and a exclusion criterion.
I am going to guess that most people that ask this question are new to programming. To be honest this should not even be a questions for any developer that knows SEO or flutter for that matter. The simple answer is that, what you are asking is for the equivalent of a Android or IOS app to be SEO. As an example for nodejs apps that use similar tech like angular, you will need a service like prerender.io to render you code to HTML to be somewhat SEO. While yes you can go around it, rendering JS for google to even rank you will be an battle on its own. Remember that Flutter, Angular, React and Vue apps, all compile to a JS file. This will make google look at the website and think it is 3-8mb. If you test a website with google insights you will realize that your website should be under 1.8mb and it has to load in under 1s to rank. SEO is not just headings and text, its is much much more. This is why something like AMP.dev makes sense. To an end the best website you can build will be done in AMP using an image optimization service like gumlet with a back end in DART or "ANGEL to make things easier". While a user might want to use your App they can than can download it in flutter. Again this is for a website that is E-commerce or similar. You also have to worry about ADA compliance that is an absolute mess to deal with in flutter. In the end if you need a site that is SEO build in AMP or HTML with back end in DART "since you guys like flutter not sure what the fuss is about using Dart". These guidelines are not made by flutter and is not that the flutter team does not want them, it is all because of search engines like Google and Bing that will rank your website based on speed, programming language, image auto size, CDNs and finally content. I hope this makes sense for any new programmers. Flutter for Web is just a container that allows you to run an Angular like app on your browser but with the flexibility of Flutter. As an example look at letgo.com these apps are not SEO by any means for that exact reason. Each framework has its own place. I honestly think this is good as we can get using Dart for server and flutter for apps and admin panels and just leave HTML for the front end. This already simplified our system by removing 4 languages from the mix, remember that before Flutter, for a website you had to do this " Node for server, HTML/Angular/CSS/React/Vue front end, Andorid JS for APP, Swift for IOS." I think we have it really well with just two languages at this point. All I can say is that if anybody has any more questions or doubts, than you can better bet that you do not know enough about web development. I do not mean this in a bad way. I mean it as, this should be your base line to realize how much you know.
I'm not sure you're right about those asking questions being new to programming. I, for one, am trying out Flutter for web because I really enjoyed programming in it, whereas I never really enjoyed Javascript. I suppose it isn't ideal (certainly not yet) and I was aware of this before I started, but it was the framework that appealed to me, that's why I started.
On the whole issue of exactly how to make it more SEO friendly, is there any chance the the accessibility features of Flutter could be used when building the website to create a more simplified (essential) layout that would load faster? Then as the website loaded, this would be replaced by the Flutter UI.
I'm not sure you're right about those asking questions being new to programming. I, for one, am trying out Flutter for web because I really enjoyed programming in it, whereas I never really enjoyed Javascript. I suppose it isn't ideal (certainly not yet) and I was aware of this before I started, but it was the framework that appealed to me, that's why I started.
On the whole issue of exactly how to make it more SEO friendly, is there any chance the the accessibility features of Flutter could be used when building the website to create a more simplified (essential) layout that would load faster? Then as the website loaded, this would be replaced by the Flutter UI.
Unfortunately that is not how google works :/ . Please check out Google insights Test . This is a test of the Shrine flutter demo app. They look at the code and what resources it loads through the entire process. So the system will load the entire application in this instance 906kb, not a lot, however the issue becomes when you try to SEO pages, they dont exists. This is because the app will load new data in the same container. So in reality you cant really get /categories/product/product1 where this is where the SEO happens. in the test you can see that google only loaded the login screen.
Unfortunately since you cant tell google how to crawl the app and since google looks for pages, than anything that is not in view is considered invisible and non existent.
This was also removed by google some times ago as people use to hide text in display none and hidden containers to try to trick the ranking for keywords that they did not actually have on the website.
*However If the app is a one page app where all the text is rendered at start than yes google will see it correctly as long as you structure it. Keep in mind that most of googles ranking is done also based on schema.org please review that.
As most of you know, you also need a sitemap that links to all pages, again flutter has no pages so that does not help.
With all that said there are way around it, please see https://nextjs.org/ they do exactly what is needed for SEO. I have asked them some time ago if they support flutter and they said NO but they are open if more people ask for it. So maybe most of you can ask them or maybe somebody will build a dart script that replicates what they do.
I can promise you that I love Flutter as much as all of you and have embraced it, but these are limitations, we learned to adapt and learn that DART can product HTML SEO code for web and flutter amazing apps for back end use and mobile devices.
Happy Coding :)
Also here is a very good resource. As you guys and girls can see this is not a flutter issue, it is a inconvenience that affects all languages of this nature that compile to JS. https://snipcart.com/spa-seo
prerender.io works pretty good with Angular and other SPA frameworks, a similar tool built for Flutter will put it in the same spot as the other frameworks in terms of SEO
@virgiltu
I'm well aware of the problems Frameworks like react, angular and now flutter have when it comes to SEO. I also know the server side rendering approaches. But your answer was not satisfying for me, so I let myself go to a slightly provocative post. Sry for that.
In the end I had the hope that a crossplatform framework from google itself at least has solutions in mind. In my opinion, the market is crying out for a framework that can be used in web, mobile & desktop in every aspect.
I can join @federicofazzeri it would be nice to have a similar approach for flutter. I will also ask nextjs for that :)
It is true you can't do anything in Flutter for SEO but you could use index.html file in web folder which helps in SEO. I mean I am using it live. So answer from personal experience. Use Meta Tags - keywords, description, title.
Hi @elaishane , Can you explain a bit more?
Thanks
Well Iβm head section there is a meta section in which it contains description. Which gives description of the website. You could form a description in a way it should Vinodβs itβs of keyword. Also there is a keyword section you could add in net tags which will help our search engines to find our website. Also you could use title
With all that said there are way around it, please see https://nextjs.org/ they do exactly what is needed for SEO. I have asked them some time ago if they support flutter and they said NO but they are open if more people ask for it. So maybe most of you can ask them or maybe somebody will build a dart script that replicates what they do.
Are they open to something like https://github.com/sponsors? I'm thinking if we can crowdfund this feature request..
The world is moving from downloaded apps from apple store and google play to PWA, installed via browsers.
Not only flutter have to think about this, but also google, bing and all others.
Google should have more ways to parse the content instead of only reading tags and barely few JS files, should have more ways to read canvas, dunno how, but would be a game changing
Also flutter could provide a SSR version that parses only on crawlers
on binary apps, flutter compiles for both IPA and APK, on web flutter could compile canvas and some SSR
I have no idea how, but one thing I know: nothing is impossible
And SEO is a must have feature, as mentioned before, imagine an e-commerce, it's impossible to have an e-commerce without SEO
and further more, flutter should also give support for google amp, json-ld and facebook instant articles
I guess speed and size are tough ones to achieve but according to https://searchengineland.com/tested-googlebot-crawls-javascript-heres-learned-220157 Google crawls inside the javascript generated content sites. What do you think guys? @virgiltu
@thibaut-d Nope, I did not. I wonder if anyone else did...
It would be super nice to get a little clarity on this from the Flutter Team.
Is this viewed as something that can even be solved? Is there a plan baking somewhere? From a pragmatic standpoint, this is the only real thing holding Flutter back from being an amazing option for crafting websites. But it holds it back in a very significant way.
Most clients won't opt for a site that can not be crawled by google, even if it's much cheaper and faster to create.
It would be super nice to get a little clarity on this from the Flutter Team.
Is this viewed as something that can even be solved? Is there a plan baking somewhere? From a pragmatic standpoint, this is the only real thing holding Flutter back from being an amazing option for crafting websites. But it holds it back in a very significant way.
Most clients won't opt for a site that can not be crawled by google, even if it's much cheaper and faster to create.
I am a bit confused why you would want to build an SEO site in flutter to start with. This is the equivalent of asking for an IOS or Android app to be SEO. Flutter for web is noting more than a wrapper for the app just like being able to run a and IOS or Android in a web-browser. While Flutter makes amazing apps it cant makes pages. For this please look at AMP project and Dart with AMP. The biggest issues with Flutter for web as a website will be that you will need to load the entire script to show one page. That is extremely wistful as google will ask that you load only the necessary code for that page to work. I hope this makes sense for everyone here. This is not a limitation that the Flutter team does not want to have it is just a rule of how the web works in general. Please watch this video to understand. https://www.youtube.com/watch?v=lBTCB7yLs8Y
Many people bundle websites as apps, or convert apps to websites, I really don't see a hard distinction between the two except that Web exists in a sandbox and has a few less privileges.
A better comparison would be SEO for a Vue or React site.
https://www.smashingmagazine.com/2019/05/vue-js-seo-reactive-websites-search-engines-bots/
Sure Flutter might not be able to make lightning fast loads like AMP, but it would be a mistake to forego "good" because you can't have "great". Many modern websites are 10mb+ in size, Flutter hello world is 2mb or so. Most React or VUE apps work the same way as Flutter, loading all the upfront 'app code' in one minified file. I can easily create nice scrollbars in Flutter that mimic the browser precisely. I don't see any hard distinction here.
Many people bundle websites as apps, or convert apps to websites, I really don't see a hard distinction between the two.
A better comparison would be SEO for a Vue or React site.
https://www.smashingmagazine.com/2019/05/vue-js-seo-reactive-websites-search-engines-bots/Sure Flutter might not be able to make lightning fast loads like AMP, but it would be a mistake to forego "good" because you can't have "great". Many modern websites are 10mb+ in size, Flutter hello world is 2mb or so. Most React or VUE apps work the same way as Flutter, loading all the upfront 'app code' in one minified file. I can easily create nice scrollbars in Flutter that mimic the browser precisely. I don't see any hard distinction here.
True but they are also not on any top pages of google. If a page does not run passed the google insights than you can consider it a huge waist of time. https://developers.google.com/speed/pagespeed/insights/ as you can see here a website has to load in less than 1s to be considered for the first page of google, resources have to be minimal at best 50kbs for JS is ideal. Remember that this is based on the customers speed where google tests 3G speeds. Yes Vue and React have SEO but they are bloated. Remember that google loads the HTML , CSS and JS in that order. In most cases Vue and React is used like Jquery so most of the site is still HTML but where some data is manipulated by VUE and React. however flutter is all JS and that does not put it in the same category. In order to make Vue and React seo you need render the html on the serve side. Kind of a pointless move as you lose the reactivity of it all. Than you have to load Vue and React again to manipulate the DOM. Sadly this is not how flutter works as it generates one JS files that loads everything. Flutter will be the equivalent to ReactNative not React and Vue,
Well in some cases, you may not have many competitors or very specific content, and it's just important that content is actually crawled. Google will still load a poorly optimized site on page 1 if it's a significantly better match. SEO is not all about ranking, it's also about just being indexable period.
I don't know in depth about how this could be possible, or what is done for React and Vue currently, which is why I posed the question, like what is the current perspective on this from the Flutter team? Is it just a Fact Of Life for Flutter? Or maybe Navigator 2.0 have some sort of support for this concept?
Certainly it seems like semantically we should be able to make this work... if we can provide some sort of robots.txt file, and ability to extract text content from individual routes, seems like there should be some ability to give google what it needs to crawl a flutter apps content and index the various routes that are exposed.
Well in some cases, you may not have many competitors or very specific content, and it's just important that content is actually crawled. Google will still load a poorly optimized site on page 1 if it's a significantly better match. SEO is not all about ranking, it's also about just being indexable period.
I don't know in depth about how this could be possible, or what is done for React and Vue currently, which is why I posed the question, like what is the current perspective on this from the Flutter team? Is it just a Fact Of Life for Flutter? Or maybe Navigator 2.0 have some sort of support for this concept?
Certainly it seems like semantically we should be able to make this work... if we can provide some sort of robots.txt file, and ability to extract text content from individual routes, seems like there should be some ability to give google what it needs to crawl a flutter apps content and index the various routes that are exposed.
I complementary understand it. Unfortunately Flutter creates a one page app and all the files are compiled into one. This is ideal for back ends, admin panels or apps that can be ran in a browser. Much like Pandora or something of sorts. Vue Js uses something like https://nuxtjs.org/ please watch the video on their front page. When it gets to seo it will make more sense. To be honest I dont think flutter was ever meant for seo. The way we use it, is to make the website with amp as minimal as possible but also provide an app like look for mobile users who wish to do so without having to install an app on their device. Or at least that is the idea. Most ideal uses for fluster for web would be in browser apps that require user login and does not get indexed. I am sure something will come along like Nuxtjs did for vuejs. In short we need a href tags and vanilla html and Flutter was never built for this. I think a framework that does that is https://angel-dart.dev/ for dart. That will be the only way to use Dart and keep the development in the same language. You will not get the same reactivity but that framework was built for this reason exactly.
@virgiltu I think a framework that does that is https://angel-dart.dev/ for dart.
I guess you mean https://angulardart.dev, which is a dart copy from frontend framework https://angular.io
https://angel-dart.dev/ is Dart on the Server which is like https://expressjs.com to node
If anyone interested, I've made up A RealWorld example app in AngularDart, according to https://github.com/gothinkster/realworld
And there is an blog about sharing biz logic within AngularDart & Flutter https://medium.com/@fandygotama/code-sharing-between-flutter-and-angulardart-198c14ec5200
I complementary understand it. Unfortunately Flutter creates a one page app and all the files are compiled into one. This is ideal for back ends, admin panels or apps that can be ran in a browser. Much like Pandora or something of sorts.
I agree. But there is also an entire category of small micro-sites where Flutter could also be quite attractive.
There are a lot of similarities between Nuxt and the routes we already have in Flutter.
If we had the ability to define some basic meta-data for a route, and to render/extract the text context for a route, seems like Flutter could do all the same stuff that Nuxt is doing. Flutter already has some degree of support for this concept with the Semantics tag.
Not to mention, the bulk of size for any site is typically the images and fonts. Often the 1mb taken up by the script file will be <10% of the actual bytes loaded. Then consider that Flutter uses 1 single connection and request for the entire site, where many websites have 30 or 40 connections, each with an overhead of 100-200ms and a max of 6 or so concurrent requests.
It's really not as simple as, Flutter loads everything in one file and that's bad.
Also, fwiw, when enabling gzip, flutter download is actually ~260 KB and downloads in around 500ms:
Roughly equal to 1 of my font files.
The thing is the that this issue has P5 priority now. Don't think dev will ever look into it soon
Why closed?
Can someone please reopen this issue? It is one of the most requested features (with over 100 likes) and it got closed without any explanations nor any solutions... :(
I was just digging into the test API, and realized that it has the ability to dive into a widget tree, find a widget of some type, and extract the textual contents of that widget.
It seems to me, if we can do this, then generating a set of crawlable html pages from a set of routes is pretty trivial?
For example, if we had a Searchable() widget, and we wrap our content in those, cant a tool just extract all the text within that sub-tree, toss it in a html page, and make that the textual representation of the route? One we can use as 'alternative content' / printable / indexable view of the route?
You could even add common SEO content types, like Searchable(type: SearchableType.H1, child: ... ) and then allow us to nest, creating semantic tags. Could do the same thing to allow us to flag links, SearchableLink(location: "details/21", child: ... ). Could also allow excluded regions with a simple Searchable(ignored: true, child: ...)
All thats left to do is generate a robots.txt, which can be done with a simple list of routes that the developer provides... Doesn't that get us really nice SEO? Am I missing something? We could sniff user agent and somehow show html text for googles crawler, but when a regular user comes to that route, they get the flutter app?
This also seems to mesh well with the new Router API which has us basically declaring all our routes anyways, not much more work to spit them out to robots.txt.
Quick experiment for fun, given a flutter page like this:
class HomePage extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
home: Scaffold(
body: Center(
child: Column(children: [
Text("Not searchable :("),
Searchable(
child: Column(mainAxisSize: MainAxisSize.min, children: [
Text("Header 1"),
Text('''Vivamus vel felis ornare, dignissim nisi nec, fermentum nunc. Sed mattis, lacus ac
porttitor fringilla, est metus scelerisque metus, sit amet sodales urna quam a dolor.
Praesent fringilla venenatis eleifend.'''),
Text("Header 2"),
SearchableLink(
"/details/24",
child: FlatButton(child: Text("Show Details"), onPressed: () {}),
),
Text('''Aliquam ornare fermentum commodo. Fusce ullamcorper ex at erat vestibulum,
ac laoreet elit posuere. Nam ullamcorper augue velit, sed suscipit sapien
consectetur eget.'''),
]))
]))));
}
}
And a test like this:
return find
.descendant(of: find.byType(Searchable), matching: find.byType(Text))
.evaluate()
.map((e) => e.widget as Text)
.toList();
}
List<T> getChildWidgets<T>() => find.byType(T).evaluate().map((e) => e.widget as T).toList();
void main() {
testWidgets('SEO Export Test', (WidgetTester tester) async {
await tester.pumpWidget(HomePage());
List<SearchableLink> links = getChildWidgets();
links.forEach((t) => print("Link: ${t.location}"));
List<Text> pageTexts = getSearchableText();
pageTexts.forEach((t) => print(t.data));
});
}
We can extract the following result:
I'm not very familiar with the api, but maybe this will inspire someone with more knowledge than me :)
How much Semantics Widget useful for SEO in flutter web ?
Looked a little closer at Semantics and ya, seems like they have everything you need to output a printable HTML content version of a Route
We could sniff user agent and somehow show html text for googles crawler, but when a regular user comes to that route, they get the flutter app?
You could but really shouldn't do that. This practice is known as cloaking and will induce a huge penalty from search engines when detected (if you don't get blacklisted forever).
The only viable approach to SEO for SPAs as of today is:
- Having the JS somehow render indexable (i.e.
<img alt="...">,<meta>,<title>,<p>, no canvas ...), hierarchical (i.e.<h{1,2,3,...}>vs<p>vs<ul><li>, ...) and standard DOM elements for everybody (no only search engines). - SSR or prerendering to optimize ranking or to support engines that don't render JS (which I don't think are that common anymore).
- Generally, use modern web standards in addition (not in place) of the standards from the early 2000's: avoid hash-based routing, have fast initial page loads, limit the amount of medias and blobs to a minimum (especially if loaded synchronously), serve relevant and consistent content (statically if possible), have metas, have a page title that corresponds to the URL and page content, have a sitemap, a
robots.txt, β¦
And you can't really workaround that without taking considerable risks. That's unfortunate. Even relying on <noscript> for SEO can sometimes be considered as cloaking (because this tag was abused for tricking search engines in the past). This means SEO support should eventually either be first-class citizen or be deliberately acknowledged as a non-goal by Flutter team (in which case it should be made clear that you should only use Flutter for portals or websites you don't expect to be indexed anyways). Put another way, I don't think it is possible to achieve proper SEO by delegating the whole work to external tools and libraries.
I can barely imagine how much work it is get proper SEO in Flutter Web (the situation is very different from React DOM for instance), and IMHO there are more important issues and features Flutter should address first. However, I also reckon that improving SEO in Flutter could be very beneficial in lots of other aspects, as working on SEO usually leads to improving loading times, rendering times and UX in general.
Thanks for the info, so technically totally possible to present a SEO version of a Flutter App, but because of pre-existing legacy issues and bad actors we can't do it as it would look nefarious. Sad.
0 is not possible with good performance. That's the reality there.
I wonder if it would be considered cloaking if a flutter site rendered with DOM for SEO's and with CanvasKit/Skia for users. That could let us present the low-fps DOM version for crawling, but present the high-fps canvas version for visitors.
That + a robots.txt + deeplink support could get us pretty damn close?
Is anyone here talking about Netlify and pre-render?
https://www.netlify.com/blog/2017/06/06/jamstack-vs-isomorphic-server-side-rendering/
I see performance stats like this and wonder if this topic is even relevant..
Big thing here, is with tools like Netlify, they implement SSR as a static host, and with 1-click pre-render option, your site will be ready for google bots
Also, now that we have navigator 2.0 we can use named routes to generate sitemap, meaning bots can now go through all the routed pages.
Thanks @gilbsgilbs and @esDotDev for highlighting the basic issue. It is now clear that without the official support of Google (and possibly of other Crawler authors), the idea of Flutter for SEO friendly web seems dead.
Just to clarify for a me: how does Google detect cloaking case? Does Google crawler to occasionally download a page under another User-agent?
Guys, hosting Flutter web with Netlify, my site has a 100% rating on search engine optimization according to Lighthouse
I am not sure if you are fully aware of what SEO is @Gene-Dana or what the real issue here is.
Obviously pages are indexed and obviously you can add meta tags.
The actual problem is that the content is not indexed. All Flutter apps are SPAs, however, they might provide access to a bunch of content. When using hyperlinks to that content, search engines like Google Search can usually index that page (i.e. the particular) page and rank that in search results.
For example:
https://example.comhttps://example.com/articles/fresh-food-supplyhttps://example.com/articles/satellite-internet
If example.com in this case is a newspaper site, these three pages would be indexed separately and the content on the different pages would be ranked by search engines. This is what SEO is about.
And this is impossible with Flutter right now because only example.com itself will indexed with the meta tags in the head and all content on the pages is ignored.
What you are suggesting is misleading, I feel. I hope what I am trying to say is understandable β€οΈ
Also, now that we have navigator 2.0 we can use named routes to generate a sitemap, meaning bots can now go through all the routed pages.
Does this not address that issue?
I means we can more easily generate a sitemap, it does not solve the problem of giving browsers something they can actually index the content of. Basically we could already do this with onGenerateRoute so it's nothing new, Nav 2 just supports more advanced routing schemes.
The much bigger problem is how can we present something to the google crawler that semantically makes sense (H1, links, etc)
@Gene-Dana not at all.
Crawlers look at the HTML. Flutter is not SEO friendly because none of the content is visible at crawl time.
Imagine usually having a <p> element with some text that would be indexed by Google Search. With Flutter, there is nothing.
Crawlers look at the HTML. Flutter is not SEO friendly because none of the content is visible at crawl time.
@creativecreatorormaybenot I'm no expert. From what I understand, Google has been crawling JS since 2015.
Also, to head off whether or not crawling JS is truly effective, doesn't pre-rendering solve this?
Big thing here, is with tools like Netlify, they implement SSR as a static host, and with 1-click pre-render option, your site will be ready for google bots
Imagine usually having a
<p>element with some text that would be indexed by Google Search. With Flutter, there is nothing.
@creativecreatorormaybenot Does the HTML renderer solve this? https://flutter.dev/docs/development/tools/web-renderers
Imagine usually having a
<p>element with some text that would be indexed by Google Search. With Flutter, there is nothing.@creativecreatorormaybenot Does the HTML renderer solve this? https://flutter.dev/docs/development/tools/web-renderers
Actually not. Both of them not show the doms of web page. Google cannot get page detailπππ
Maybe the dart language is "Born to beat js"? I think this is the best chance... Just my idea, dev speed of Flutter is faster than any js framework(Vue.js, React, Angular.js..). We all hope official can focus on this.
I only have one page on this site. I would like to create another site, submit a sitemap, and see what it says
Actually not. Both of them not show the doms of web page. Google cannot get page detailπππ
Let's truly test it here. So far, this site does show up on google
Actually not. Both of them not show the doms of web page. Google cannot get page detailπππ
Let's truly test it here. So far, this site does show up on google
Agree with above, I'm not sure you are deep inside seo... Crawler grab the dom of html. For now, Flutter web just support like <iframe>. This's not good for seo. Crawler cannot grab content from a big js file...
We need a solution to make every page like:
<html>
<body>
<h1>Flutter web<h1>
<p>good for seo</p>
<p>we need flutter web seo!</p>
</body>
</html>Google can get content or keywords like "Flutter web"γ "good for seo" or "we need flutter web seo!". Then people can get result by input keywords.
but not:
<html>
<script src="main.dart.js" type="application/javascript"></script>
<body>
<flt-glass-pane>
<flt-semantics-placeholder></flt-semantics-placeholder>
</flt-glass-pane>
</body>
</html>Google get nothing from html ! People can only search website by website title!
What do you think about this? https://github.com/johnpryan/tavern
What do you think about this? https://github.com/johnpryan/tavern
This tool is for legacy Dart web, not Flutter Web.
We could sniff user agent and somehow show html text for googles crawler, but when a regular user comes to that route, they get the flutter app?
You could but really shouldn't do that. This practice is known as cloaking and will induce a huge penalty from search engines when detected (if you don't get blacklisted forever).
The only viable approach to SEO for SPAs as of today is:
- Having the JS somehow render indexable (i.e.
<img alt="...">,<meta>,<title>,<p>, no canvas ...), hierarchical (i.e.<h{1,2,3,...}>vs<p>vs<ul><li>, ...) and standard DOM elements for everybody (no only search engines).- SSR or prerendering to optimize ranking or to support engines that don't render JS (which I don't think are that common anymore).
- Generally, use modern web standards in addition (not in place) of the standards from the early 2000's: avoid hash-based routing, have fast initial page loads, limit the amount of medias and blobs to a minimum (especially if loaded synchronously), serve relevant and consistent content (statically if possible), have metas, have a page title that corresponds to the URL and page content, have a sitemap, a
robots.txt, β¦And you can't really workaround that without taking considerable risks. That's unfortunate. Even relying on
<noscript>for SEO can sometimes be considered as cloaking (because this tag was abused for tricking search engines in the past). This means SEO support should eventually either be first-class citizen or be deliberately acknowledged as a non-goal by Flutter team (in which case it should be made clear that you should only use Flutter for portals or websites you don't expect to be indexed anyways). Put another way, I don't think it is possible to achieve proper SEO by delegating the whole work to external tools and libraries.I can barely imagine how much work it is get proper SEO in Flutter Web (the situation is very different from React DOM for instance), and IMHO there are more important issues and features Flutter should address first. However, I also reckon that improving SEO in Flutter could be very beneficial in lots of other aspects, as working on SEO usually leads to improving loading times, rendering times and UX in general.
From this article on Dynamic Rendering It doesn't seem like this approach would be considered cloaking as long as the HTML served closely resembles the data actually shown. Does anyone have input on this / could anyone clarify the line? This seems like a big grey area.
Agree with above, I'm not sure you are deep inside seo... Crawler grab the dom of html. For now, Flutter web just support like <iframe>. This's not good for seo. Crawler cannot grab content from a big js file...
We need a solution to make every page like:
<html> <body> <h1>Flutter web<h1> <p>good for seo</p> <p>we need flutter web seo!</p> </body> </html>Google can get content or keywords like "Flutter web"γ "good for seo" or "we need flutter web seo!". Then people can get result by input keywords.
but not:
<html> <script src="main.dart.js" type="application/javascript"></script> <body> <flt-glass-pane> <flt-semantics-placeholder></flt-semantics-placeholder> </flt-glass-pane> </body> </html>Google get nothing from html ! People can only search website by website title!
i wonder, is it possible to automatically modify the generated html of flutter web to attach all string-user-readable-data into the html? which also will be replaced by the canvas anyway at the runtime. That being said, google can crawl the html data and user can see the same data from canvas
In fact, I hope you can share the optimization skills about SEOοΌThis problem has been bothering us for a long time, and there is no good way to solve it. I also want to ask whether the flutter team has an optimization plan
I've been following this thread for a while and took inspiration from @emmett-deen to look into dynamic rendering. I have a solution for my Flutter Web app now and it only took a few days to implement. While this will not work for every Flutter app and is also not automated or a real solution to the problem, I hope I can help some of you.
I'm currently supporting two minority founders who have built a Flutter real estate app for their native Honduras, sort of like a simplified Zillow.com. Given the limited resources it's important for them to have just one codebase and thus they're also using Flutter Web. As you can imagine SEO is crucial however since all those properties won't be found organically otherwise.
There are 3 main pages for this app that need indexing, the other ones, like seller analytics, uploading a property etc aren't important for SEO. The pages are:
- List of properties
- Property detail page
- Seller profile with seller properties
So I have very few sites that are actually important for SEO. I stumbled across this Google Firebase video:
https://www.youtube.com/watch?v=LOeioOKUKI8
And given my setup with Firebase Functions and Firebase Hosting, it inspired me to do the following:
- Find an open source html5 real estate template so I don't have to put in time to develop something only bots will see
- Use Handlebars to generate those 3 pages as shown in the video
- Use Firebase Functions with Express x Hosting together as shown in the video, with a simple "if" check, that supplies regular users with the Flutter Web app and bots with the html5 app. Important here is to make sure all links are actually the same between html5 and Flutter web
- Basic SEO improvements like caching, metadata, robots.txt, sitemap.xml
- Afterwards request indexing from Google Search console
As @emmett-deen mentioned, by making sure the data is the same, this should not be considered cloaking.
Now my SEO ranks 95+ on most pages and I can expect good results as Google indexes my site in coming weeks. Again, I understand this won't work for everybody but it took me a weekend to get this result that may help others. Leave a reaction here if you think I should turn this into a proper tutorial.
@FloPMT from your explanation it does sound like an automated system of the Google search crawler might flag this a cloaking π€
I would highly recommend checking out this article on cloaking to verify if that could be considered cloaking by a human AND more importantly if a bot could also figure out that IT'S NOT CLOAKING π€
Depending on how you set it up either might be true. Just want you to be aware and not surprised that you may get punished by the Google search algorithm for something that you did not intend to do.
This may help as well:
https://developers.google.com/search/mobile-sites/mobile-seo/dynamic-serving?hl=en
I've been following this thread for a while and took inspiration from @emmett-deen to look into dynamic rendering. I have a solution for my Flutter Web app now and it only took a few days to implement. While this will not work for every Flutter app and is also not automated or a real solution to the problem, I hope I can help some of you.
I'm currently supporting two minority founders who have built a Flutter real estate app for their native Honduras, sort of like a simplified Zillow.com. Given the limited resources it's important for them to have just one codebase and thus they're also using Flutter Web. As you can imagine SEO is crucial however since all those properties won't be found organically otherwise.
There are 3 main pages for this app that need indexing, the other ones, like seller analytics, uploading a property etc aren't important for SEO. The pages are:
- List of properties
- Property detail page
- Seller profile with seller properties
So I have very few sites that are actually important for SEO. I stumbled across this Google Firebase video:
https://www.youtube.com/watch?v=LOeioOKUKI8And given my setup with Firebase Functions and Firebase Hosting, it inspired me to do the following:
- Find an open source html5 real estate template so I don't have to put in time to develop something only bots will see
- Use Handlebars to generate those 3 pages as shown in the video
- Use Firebase Functions with Express x Hosting together as shown in the video, with a simple "if" check, that supplies regular users with the Flutter Web app and bots with the html5 app. Important here is to make sure all links are actually the same between html5 and Flutter web
- Basic SEO improvements like caching, metadata, robots.txt, sitemap.xml
- Afterwards request indexing from Google Search console
As @emmett-deen mentioned, by making sure the data is the same, this should not be considered cloaking.
Now my SEO ranks 95+ on most pages and I can expect good results as Google indexes my site in coming weeks. Again, I understand this won't work for everybody but it took me a weekend to get this result that may help others. Leave a reaction here if you think I should turn this into a proper tutorial.
This is exactly what I was thinking! Please keep us updated on if this gets flagged as cloaking!
Yes but for normal users we would serve:
<script src="./main.dart.js">For crawlers we would serve:
<div class="house" id="house1">
<p>Beach house</p>
<img>
</div>
<div class="house" id="house2">
<p>City appartment</p>
<img>
</div>Which to a bot is completely different, but to a human obviously is the same visually.
Yes but for normal users we would serve:
<script src="./main.dart.js">For crawlers we would serve:
<div class="house" id="house1"> <p>Beach house</p> <img> </div> <div class="house" id="house2"> <p>City appartment</p> <img> </div>Which to a bot is completely different, but to a human obviously is the same visually.
@IchordeDionysos what if we provide both of them inside the html ? like
<p>string1</p>
<p>string2</p>
<script src="./main.dart.js">
so the string1 and string2 will be visible for bot, but will be replaced by the dart.js anyway but still string1 and string2 will visible for user
That might work π€
I've also read some suggestions that this could be put into the <noscript> tag.
Disclaimer: Those were suggested like 10 years ago :D So not sure if that still applies?!
Yes but for normal users we would serve:
<script src="./main.dart.js">For crawlers we would serve:
<div class="house" id="house1"> <p>Beach house</p> <img> </div> <div class="house" id="house2"> <p>City appartment</p> <img> </div>Which to a bot is completely different, but to a human obviously is the same visually.
@IchordeDionysos what if we provide both of them inside the html ? like
<p>string1</p> <p>string2</p> <script src="./main.dart.js">so the
string1andstring2will be visible for bot, but will be replaced by the dart.js anyway but stillstring1andstring2will visible for user
I think the issue with this would be during that brief loading period they would see a raw HTML page with those strings which would be a less than ideal experience, however, this would work to at least include metadata tags which would still help some with SEO & would fix the sharability problem.
@Hixie I wonder if the Flutter team could increase the priority of this issue, As it is really important for startups who uses Flutter web
Thanks a lot
@Hixie I wonder if the Flutter team could increase the priority of this issue, As it is really important for startups who uses Flutter web
Thanks a lot
What you see being investigated here is navigation, one the main features that needs to be addressed before SEO can be achieved.
https://github.com/flutter/uxr
Without a clear navigation scheme, crawlers will have a problem crawling.
I would go out on a limb and say that navigation is a top priority, which means SEO is too
What you see being investigated here is navigation, one the main features that needs to be addressed before SEO can be achieved. https://github.com/flutter/uxr
Without a clear navigation scheme, crawlers will have a problem crawling.
I would go out on a limb and say that navigation is a top priority, which means SEO is too
@Gene-Dana
I hoped the whole web roadmap would have been up to discussion flutter/uxr#52
I don't see how router implementations in any way are blockers for discussions on SEO strategies. As long as you can have a path, that leads to a page, then we can at some level implement a "crawlable" site, we've been able to do this using onGenerateRoute for a long time.
UXR will recommend some packages to make the declaring of your routes more flexible or readable, but the core problem persists, that has always persisted, of how to show those routes to a search crawler. The semantics widget seems like the obvious path forward, but would be cool to know if the team even sees this as a viable option.
I have a kinda hack that seems to work. Visit coronaflash.com and turn your user-agent to βgooglebot/β. You see a traditional DOM appearing on top of the app, moving in-sync with the app. Google Search seems to like it.
What I am doing: If I detect a crawler/bot, I turn the render tree into a text tree at the right positions and add it to the DOM on top of the flutter glass pane.
Search yields:
@derolf Looks good. Appreciate if you could share the code.
@derolf I am interested in how you added your text tree to the DOM on top of render tree.
I really hope Flutter team gives it more time/work to handle soon
I have a kinda hack that seems to work. Visit coronaflash.com and turn your user-agent to βgooglebot/β. You see a traditional DOM appearing on top of the app, moving in-sync with the app. Google Search seems to like it.
@derolf are you using flutter_html ?
@derolf I think Google SEO might have punished you because I do not get your site as a search result unless I explicitly include your site name in the query.
Would be nice to have for Flutter something like Next.js for React.
Try Corona vaccine finder Berlin.
@derolf I think Google SEO might have punished you because I do not get your site as a search result unless I explicitly include your site name in the query.
@derolf Yeah, no luck unfortunately. From what I can tell, it depends on the user searching, i.e. I am seeing different results being signed in to Google and not being signed in.
In a private window, the best I can get is the main app (so not the separate route) showing up on position 4. And according to Google, I am even searching from Berlin, so that should not be the issue, I think.
@derolf are you able to provide an example of how you did this?
I am not sure how @derolf have achieved this but I tried something and created a temp plugin if anyone want to use it.
https://github.com/iamSahdeep/seo_renderer
https://pub.dev/packages/seo_renderer
Basically just using renderObject to get absolute position and adding DivElement in case navigator have a bot as userAgent using RegEx.
Got the same results as derolf.
If it actually works for SEO we can maintain it till Flutter Team creates a concrete solution for us.
Hello Guys:
I was wondering if we controlled the index.html via SSR (something like jsp thymeleaf or expressjs) to render seo friendly html that would then be overridden by the main.dart.js.
I for instance developed https://rapidoc.app using flutter. it is basically a website that allows people to book for doctors.
I have a doctor list page that would be nice if it gets indexed (the doctor list page)
I am pretty sure that generating the seo friendly html under the flutter up will work (ui wise) but am not sure if it would work seo wise.,
@Oualitsen I believe that's an option and that's called dynamic rendering: https://developers.google.com/search/docs/advanced/javascript/dynamic-rendering
However, it's unsure if this would flag your website for cloaking at this point.
@Oualitsen I noticed your website loads pretty fast. Was it really built with flutter? If so how did you achieve this?
k guys let's move on from non-flutter-web of @Oualitsen
just focus on finding ideas how to implement SEO in flutter web
@nivla360 the website is not written with flutter.
Let me explain:
The link I shared is a kind of a landing page of a whole project.
That landing page is basically html 5. But the real work (booking and searching) was made in flutter.
I have a database of doctors that I want to be indexed by Google but keep it under the flutter app. this feature is developed yet so I was wondering if my idea was good.
So yeah the link I shared is plain html5.
just looking at this, there is mention of SEO, but nothing specific.
https://medium.com/flutter/flutter-whats-next-on-the-web-e0454bff964
I hope a solution comes from the Flutter team shortly.
I'm not sure if a hack like https://pub.dev/packages/seo_renderer is a proper solution since it could even hurt the SEO since it is a hack Β―_(γ)_/Β―
Looking forward for official solutions.
Any news on this?
I have e-commerce project and the client want app & website (same requirements exactly)
build the web with flutter ?? [ save money and time to share same code ]
or hire a programmer to made web with any other framework node.js || Vue.js
please help me to choice
@Obada2020 Hi, I've recently built website, android & ios app with flutter. So in my opinion use flutter to built android and ios app. For website don't use flutter you'll regret later on(as I'm). I'm also going to re-write a website using html-css-js so that it can load fast, have better seo, can integrate facebook messenger, can claim it on pinterest and the list goes on. For website flutter is far from an alternative as of now. And for android and ios apps it is the best thing to use right now.
@KartikShrivastava I recently tried Sveltejs in a project. Itβs outstanding and also provides built-in SSR. Give it a try!
Ya, really no point in using Flutter for anything that wants to sell products and also be on the web. Once you have two code bases, the entire value proposition of Flutter goes out the window.
Svelte + Typescript looks awesome, and bundle that into a native app with Capacitor or something (https://capacitorjs.com/).
Just to confirm (in case I'm missing something): Svelte js isn't actually relevant to Flutter SEO, right?
It's as relevant as bringing up Kotlin, C# or Swift when discussing dart features, imo. Just a competitor in the same space, that can be looked to for reference.
Also, for developers who find this issue, see that the official response is no response, they might want to know of some alternatives.
Ok, makes sense. Just wanted to make sure as I couldn't see any way to use it with Flutter. But yes, if you're thinking of starting a project, it is an interesting option.