Page 1 of 1

searchtempest not feeding RSS?

Posted: Tue Dec 13, 2016 3:28 pm
by norkeel
Newbie here, 4 days ago i set up and saved 4 craigslist searches (approx 40 cities) with searchTempest, then set up a feedly account to display these results. Looked great.
I noticed today that ST picked up a new CL posting that is not shown in Feedly. Hmm.

Forgive the basic questions, but how should this work?
When i open Feedly should it kick off a new ST search? If not, do i need to go to ST and click the search button for each saved search to update the OPML file?
Shouldn't ST update feedly after every saved search is executed? I would like to have my 4 searches execute in ST and update in Feedly everytime i open Feedly.
Auto-search and notification of new postings would be awesome.

Sorry for my primitive understanding, your help is much appreciated.
BTW, I love ST and am so excited to have found it.

Re: searchtempest not feeding RSS?

Posted: Tue Dec 13, 2016 3:49 pm
by Tempest Nathan
Great questions! Hopefully I can clear everything up.

Basically what the OPML file is, is a list of links to RSS feeds published by craigslist (and one from eBay if you have eBay results turned on and are searching in the for-sale category). It's formatted in a standard way that can be imported to RSS readers. Once the Reader imports the OPML file though, it then accesses those RSS feeds directly. So after the OPML file is imported, searchtempest is completely out of the picture. It's no different than if you'd gone and grabbed all those feed links and imported them manually. Feedly gets the results directly from craigslist and ebay.

So.. all should be well, except that craigslist's feeds are notoriously unreliable. As far as I can tell without having any insider knowledge there, they have some very aggressive throttles set up, so that any server that accesses their feeds 'too much' gets blocked for a period of hours to days. Since all the large RSS readers handle thousands to millions of feeds, they all inevitably get blocked from time to time. Sometimes you can have better luck with a paid reader like NewsBlur, since it will have fewer users, and so will be accessing craigslist less often, but there will still likely be outages.

So, unfortunately the RSS feeds work great when they work, and shouldn't miss any results. But sometimes they won't work for a while. Alternatively you can just search using SearchTempest. Our default results (at least for now) are powered by Google. That tends to be pretty reliable in terms of always returning something.. but it often doesn't return everything. There is generally a delay before Google picks up new posts, and some it will miss entirely.

Finally, SearchTempest has a "Direct Results" option, where we basically generate links directly to craigslist results pages. This ensures that results are complete and up to date. But it also means you have to click through each city that you're interested in. Faster than doing it manually, but still takes a while. Fortunately, we're working on a much improved version, which we will be releasing in the next day or two! This new version will combine the results for each group of 20 or so cities into a single link, so that you can cover a standard 400 mile search radius in only a couple clicks, and even the entire USA is only 20 or so. We expect this will be the best tradeoff for most people. As soon as it's ready, you should see a notice at the top of the results page with more info.

Please let me know if you have any questions!

Re: searchtempest not feeding RSS?

Posted: Tue Dec 13, 2016 4:37 pm
by norkeel
Ahh, that helps alot, great to know. Thanks.

One follow-up question. Let's say I want to change any search criteria - I will need to replace the entire OPML file right?
I'd do this by updating the search criteria in ST and run, then save the search. ST will generate a new list of links, and package them in a (newly named?) OPML file. I then need to import this into Feedly, and remove the old one. Is this right?

Thanks again for your fast and thorough answers!

Re: searchtempest not feeding RSS?

Posted: Tue Dec 13, 2016 4:45 pm
by Tempest Nathan
Yes, that's exactly right. Since the feeds are out of our control once the OPML file is created, the only way to update them is to generate a new file and replace them, just as you described.