We have never been able to get Google to index every listing page on our marketplace.
We would like to have all listings in our marketplace indexed by Google, so if a user is searching for that item, it has the chance to come up in the search results. We use Google Search Console but no matter what we do, all the detailed listings are not getting indexed.
Obviously having Google index all listing pages would be a huge plus for all of us. Any information or pointers to achieve this even if it means custom programing would be greatly appreciated. Thanks! - Susan
Hi Susan,
I’ve also had the same experience. Going to Google.com and entering “site:https://search.postcardtree.com” shows 3240 pages but there are 40k listings at least. Originally Sharetribe didn’t generate a sitemap.xml file automatically (which is what search engines use to index). This was discussed a bit at https://www.sharetribe.com/community/t/how-to-add-a-sitemap-xml-file-to-your-sharetribe-marketplace/190. I believe that the sitemap.xml is now created automatically but I concluded there must be a limit in place for how many pages/listings are actually included. I ended up trying to generate the sitemap.xml myself with external software and then uploading it to Sharetribe (see https://search.postcardtree.com/sitemap.xml). This file doesn’t appear to have updated since Oct 22 which indicates to me that Sharetribe isn’t keeping it updated. Curious to hear what others experiences are. My site is still running 10.0.0.
Thanks, Neil
Indeed, getting all the pages of a website indexed can be quite challenging, especially for large websites. It’s likely that you will never get to 100% index.
You’re right, Google (and other search engines) use sitemaps as one of the signal to discover pages. Sharetribe Go generates a sitemap automatically (this feature was added back in September 2016) and has been working smoothly since. It lists the 500 more recent listings of a marketplace. You can learn more about it at https://help.sharetribe.com/en/articles/674130-sitemap-xml-file.
With the 500 more recent listings, and regular crawls of your website by Google bots, this should be enough to let Google know of all pages in your marketplace. However, this is not all: Google can still decide not to index listings that it has discovered, for plenty of varied reasons. In addition, Google also discovers and decides what to do with pages based on links pointing to the page, your website authority, etc.
You can also learn, from Google, the reasons why some of your pages are not indexed, thanks to Google Search Console (in “Indexing / Pages / Why pages aren’t indexed”). This can be helpful to notice if something is very wrong with your setup!
We have a valid sitemap. The real issue we are having is that two thirds of our listings are never indexed by Google because they are reporting back they have a status of “Alternate page with proper canonical tag”.
We have a niche marketplace with a lot of listings. So we often get listings urls that look like this:
These are actually two different listing for different version of a similar item with different listing descriptions. Other times we have two different sellers selling the same item with a slightly different description.
This is a major issue for us as more that half our items are not listed in the search engines. Any ideas on how to best solve this would be greatly appreciated. Thanks - Susan