This forum's search feature is weak, at best. So, for years, I've instead used Google as my search engine for the forum by adding "site:themagiccafe.com" to the search query. It worked beautifully. That is, until fairly recently. Now it is rare to get a hit (that I'm looking for).

Has the forum code been changed to prevent Google bots from crawling the forum?

Also, there's a site called brotts.com that appears to be duplicating the content from the Magic Cafe. If crawling the Cafe with bots has been shut off, is brotts.com the reason for the change?

Mark

When you get that, change "Brotts" in the url to "Themagiccafe" for a workaround.

Mark, I've noticed the same thing. I received an email over the weekend from someone wanting to buy my Entertainer's Business Toolkit book. He wanted to read some reviews first so he googled the Magic Cafe Entertainer's Business Toolkit and received the Brotts site. When clicking on the link he was asked to login and it told him he didn't have access to the content.

This was the first I've heard of all this. I thought maybe it was a torrent site?

It's a test site for site builders.

Mark, nothing has changed on the Cafe but here's a recent topic about the bott site. https://www.themagiccafe.com/forums/view......forum=50

Basicaly it's a site that's used to test changes and edits to the Cafe and is only available to the techs.

Yeah, I'm getting very, very few, if any, search results when I search in Google. Very strange. My wife does Search Engine Optimization for her job and says that it probably has to do with outdated backend site maintenance of the Cafe and hasn't kept pace with Google.

Yep, previously, when I used Google to search for, say, "Chinese Sticks site:themagiccafe.com", I would get dozens of hits for various pages in the Magic Cafe forum. I just did that search now and got a SINGLE hit and it was for the word "Chinese", not for "Chinese Sticks".

I am quite confident the word "Chinese" appears hundreds of times in this forum. So, a single hit?!

Clearly, SOMETHING has changed (whether it was intentional or not) with how Google is crawling the site. I guess I'll try a different search engine.

Mark

Yes, search engine makes a huge difference!!

Google's results:



Bing's results:



Obviously, I'll be using Bing from now on.

Mark

Mark, the search engine DuckDuckGo also gets good results (similar to your screenshot from Bing)

DuckDuckGo has some privacy advantages as a search engine.

https://duckduckgo.com/about

I still can not search anything on google, I tried all these methods. This is a serious change for the popularity of magicafe

Did you try the DuckDuckGo search engine ? https://duckduckgo.com/about (*public service announcement: you should regard Google as more or less spyware . Avoid Google if possible. Use a privacy-enhanced search engine like DuckDuckGo which does not track you , and does not collect and sell your personal information.)


Specify the search results using search syntax site:themagiccafe.com

For example, if you are seeking information on the Linking Rings effect , enter your search as linking rings site:themagiccafe.com

This will return dozens (hundreds) of search results only from themagicafe.com




or a search for Dai Vernon site:themagiccafe.com will return only results on topics relevant to, or mentioning the name of , Dai Vernon.




NOTE: these screenshots I posted only show what is visible on my browser screen when I take the screenshot, but when I scroll down there are many search results, all from the domain themagiccafe.com , not from other domains. You must include the tag site:themagiccafe.com after your search term. Like this: Half-dyed Hank site:themagiccafe.com .




If you don't get search results after doing this , then you have some sort of blocker installed in your browser or on your computer or a blocker from your Internet Service Provider which is preventing you from freely using search engines. Try using a VPN (virtual private network)service to search. Windscribe is a good one. It's free to use for up to 10GB of data monthly, or if you need to use it for more than 10GBs of data there is a paid version which costs $9.00 U.S. per month if you pay by the month or that cost comes down to $5.75 U.S.D. per month if you pay by a one-time payment of $69.00 U.S.D. per year. For my own usage of a VPN (which is seldom) I have never exceeded 10GBs per month, so I just use the free version). https://windscribe.com

.

I managed to use the duckduck go site but still the results are fewer comparing the results I was getting from google. It hasn’t the same searching power. I can’t understand why the moderators of the cafe they allowed this problem to happen.

I don't think the moderators or owner of the forum had anything to do with it.

Mark

That is correct. As I stated above, nothing has changed on the Cafe. We control the search function on the Cafe but have nothing to do with outside searches. We do understand the Cafe search function was impacted when we lost our servers last year and did our best to recreate it as much as possible

I noticed the same thing.... Whenever I wanted to see a thread on the cafe about a new effect I just googled: "magic cafe sonore" ... and the thread would pop up ... Now that does not happen... even if I use Duckgo... this thread will not show...

And if I use the search function on the cafe itself also nothing comes up .... At least not for "SONORE"....

I may have found the answer. Google requires a robot.txt file to allow Google search to crawl the site. It looks like a feature so that Google search does not overload a server unintentionally. Information can be found here:

https://developers.google.com/search/doc......ts/intro

Maybe the cafe admnins can look into this.

It looks like robot.txt has the following:

User-agent: Baiduspider
Disallow: /

User-agent: 360Spider
Disallow: /

User-agent: Yisouspider
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: Sogou web spider
Disallow: /

User-agent: Sogou inst spider
Disallow: /

User-agent: claudebot
Disallow: /




To allow the GoogleBot, I think you need to add the following to robot.txt:

User-agent: Googlebot
Disallow:

User-agent: AdsBot-Google
Disallow:

User-agent: Googlebot-Image
Disallow:

You don’t have to proactively allow Google bot necessarily, although it can’t hurt. My best guess is that Google’s ranking algorithm removed everything when the cafe was down and hasn’t attempted a crawl to reindex the site. Fixing this is a matter of using Google’s free tool called Search Console and providing an XML site index file. As long as the site isn’t in the Google penalty box for some reason, it will get reindexed promptly. If it is in the penalty box, there are tools and processes in Search Console to overcome that. I do this work for a living (to pay for my magic addiction). If anyone wants to reach out to me, I’ll be happy to help pro bono.