Latest CL postings not included?

Archived. Please post suggestions to our new community page!
Locked
hhkiwi
Posts: 2
Joined: Tue Sep 10, 2013 12:14 pm

Latest CL postings not included?

Post by hhkiwi » Tue Sep 10, 2013 12:21 pm

Hi, I am new to the forum and let me start by saying that I love SearchTempest. It's a great idea and I use it on a regular basis.

There are two issues though I find irritating but I suspect that both issues have the same root cause:

1. Many results are outdated, i.e. when you click on them you get a "page not available" or "posting deleted/removed" message; and

2. More importantly, it appears to take several days until items posted on CL appear in ST searches. For example, on Orange County (CA) CL there's a posting for a motorcycle originally posted on 09/08: http://orangecounty.craigslist.org/mcy/4055304554.html. When I search for the terms "bmw xmoto" (both terms are in the CL listing header) ST won't find the listing. That's assuming I am not making a stupid mistake...

Thanks!!

User avatar
Tempest Nathan
Site Admin
Posts: 1371
Joined: Fri Apr 18, 2008 10:06 pm
Contact:

Re: Latest CL postings not included?

Post by Tempest Nathan » Tue Sep 10, 2013 1:14 pm

Yes, you're right, the reason for both is basically that our default results mode is powered by Google, and there is some delay before their index is updated. It ranges from a few minutes to a few days usually. However, removed posts can take longer since unlike new pages, Google has no way of knowing they're gone immediately. These answers in our FAQ have more details, and list a couple of alternative modes you can use to avoid these issues:

http://www.searchtempest.com/faq.php#missing
http://www.searchtempest.com/faq.php#deleted

Hope that helps!

hhkiwi
Posts: 2
Joined: Tue Sep 10, 2013 12:14 pm

Re: Latest CL postings not included?

Post by hhkiwi » Tue Sep 10, 2013 5:17 pm

Thanks Nathan. This may be a silly question but why is it not possible to automate the direct results mode so that it is not necessary to manually click "next" 100 or more times to go through all CL sites?

User avatar
Tempest Nathan
Site Admin
Posts: 1371
Joined: Fri Apr 18, 2008 10:06 pm
Contact:

Re: Latest CL postings not included?

Post by Tempest Nathan » Tue Sep 10, 2013 8:57 pm

Well, basically the Direct Results mode is just a bunch of links to craigslist results pages. We've added a bit of fanciness to make it more convenient, having the index and the next button, but essentially it's just links that open in a new window. So the only thing we could do to show more than one city at a time would be to open a bunch of new windows at once. We decided that being able to flip through the cities quickly using the next button would be more convenient than having potentially 100 separate new windows open up.

We used to have it set up so that you could view multiple results window embedded within one large window, using what are called 'iframes', but about a year ago craigslist decided to stop allowing their pages to be shown in iframes. That's when we developed the Google mode and the Direct Results mode as it now exists.

camroncamera
Posts: 2
Joined: Sat Jan 25, 2014 4:04 pm

Re: Latest CL postings not included?

Post by camroncamera » Sun Jan 26, 2014 12:41 am

hhkiwi wrote:Thanks Nathan. This may be a silly question but why is it not possible to automate the direct results mode so that it is not necessary to manually click "next" 100 or more times to go through all CL sites?
BUMP
Hi, I realize this thread is several months old. I truly miss the "old" Combined Search Results mode, but I understand that that is the reality of the situation. I believe the overall usefulness of SearchTempest has been decimated since Craigslist enacted those changes. Expired posts that still appear, and especially, new listings not showing up, just make me mistrust Google's ability to give reliable Combined Results.

I am thankful to have the Direct Results mode, but hhkiwi has expressed my frustration with that mode as well. It takes about 20 minutes for me to individually click "Next" and view through each Craigslist location in the country, for each search term I want to try. Before, I had accurate up-to-the-minute results on any search term in one click. We all have busy lives, and I just don't have the time to Direct Results search every day... unfortunately I have missed out on some Combined Results Craigslist listings in the past year due to Google's slow indexing - listings that may not have gotten away had I had a faster, easier time with Direct Results mode.

Like hhkiwi suggests, I would find a great deal of value in some automation of the "next" button in Direct Results mode. The way I picture it in my head, I would have an optional dialog box where I can set the number of browser tabs that are allowed to be opened at one time on the search results browser window (5, 10, 25, no limit, etc.). When the user hits the Search button, the search window opens all the Craigslist cities with that number of allowed tabs all at once. The user is freed from having to manually press "Next" hundreds of times, as well as from having to wait seconds for realtime results to appear before pressing "Next" again. For most of the search terms that I use, the majority of cities have zero results. In addition to the dialog for "Number of Allowed Results Tabs", I think this feature would be greatly enhanced by an "Automatically Close Tabs with Zero Results" checkbox. With this dialog activated, as the name suggests, all direct results tabs that automatically open which have "0" (zero) Craigslist results are automatically and instantly closed.

My suggested feature would not be quite as good as the "old" Direct Results mode:
Each Craigslist location search result will remain on its own individual tab rather than have all Craigslist location results in one window - however, with the addition of a "Automatically Close Tabs with Zero Results" checkbox, users may have very few tabs to examine in the end for some search terms.
The user will still have to manually click through each results tab which has nonzero hits - however, I can close one tab that I have finished examining and see the next preloaded tab instantly appear in it's place more rapidly than waiting for a realtime search when hitting the "Next" button in the current implementation. Most users can probably quickly view and close two or three Craigslist tabs that do not contain the desired listings, in the time it can take for a single realtime Craigslist search.

Even the addition of an "Automatically Skip to the Next City for Pages with Zero Results" checkbox in the current implementation would be valuable in my opinion.

I hope you've found this feedback useful, and thanks for listening :)

User avatar
Tempest Nathan
Site Admin
Posts: 1371
Joined: Fri Apr 18, 2008 10:06 pm
Contact:

Re: Latest CL postings not included?

Post by Tempest Nathan » Sun Jan 26, 2014 12:49 am

Feedback is always appreciated! Unfortunately, your main suggestions involving detection of cities with no results... are not possible. We literally have no way of seeing what is in the window the comes from craigslist. (It's a browser security measure. You wouldn't want one site being able to see what you're doing on a completely different site, right?) So there's really nothing we can do there.

Theoretically we could do something that launches a bunch of tabs at once, but I don't think we're going to. The majority of people who try to use DR now are already somewhat overwhelmed by it, and we don't want to make it worse. I do understand the value for power users like yourself though, and I apologize. That said, we do have a power option that I think will work much better for you: our RSS Feeds tool. If you're looking through hundreds of cities, especially every day, this will save you boatloads of time compared to Direct Results, and shouldn't miss anything. Check it out: http://www.searchtempest.com/faq.php#rss

camroncamera
Posts: 2
Joined: Sat Jan 25, 2014 4:04 pm

Re: Latest CL postings not included?

Post by camroncamera » Tue Jan 28, 2014 2:12 am

Tempest Nathan wrote:Feedback is always appreciated!
First off, THANK YOU for the incredibly speedy reply! Amazing to see such a thoughtful reply in such a short time :)
Secondly, my goodness I didn't mean to write such a long reply, but I hope you can bear with me here and see if there is any useful logic in any of the following scenarios. Unfortunately, I am not a web developer and have no knowledge of the technical side of how this stuff all works. :D
Tempest Nathan wrote:Unfortunately, your main suggestions involving detection of cities with no results... are not possible. We literally have no way of seeing what is in the window the comes from craigslist. (It's a browser security measure. You wouldn't want one site being able to see what you're doing on a completely different site, right?) So there's really nothing we can do there.

Theoretically we could do something that launches a bunch of tabs at once, but I don't think we're going to. The majority of people who try to use DR now are already somewhat overwhelmed by it, and we don't want to make it worse.

Okay I think I can see your point on the security issue. I was wondering to myself as I was thinking about these feature suggestions whether a site that could automatically open a series a tabs (potentially a hundred or more, if allowed :shock:) would be a security issue. But, I figured if the current implementation has the power to open two new windows, formatted side-by-side, it seemed reasonable to me that there was no greater security risk by adding additional tabs to one of those windows. I did not really think how the security issue was really more about one site being able to see what you're doing on a completely different site, and I am glad you have clarified that.

I think what you are saying is that, in Direct Results Mode, when the user types in a search term, SearchTempest isn't looking at that search term... that, in fact, the search term box isn't really used by SearchTempest at all, other than as a text placeholder. Direct Results Mode simply shoots whatever text is written in that box over to the next Craigslist city page when the User hits Next City, then the next, then the next, and so on. Each sequential Craigslist city is fed that text in the Direct Results search term box. Then each Craigslist page performs its own search using the term written in that box, but SearchTempest doesn't actually do anything with the search term, other than conveniently and temporarily hold it there for the user. Am I even close on this? Am I even making any sense? lol

Did the previous implementation of Combined Results Mode (back when there was no "iframes" restrictions) in fact "see" a user's search term? Was the search term used directly by SearchTempest to generate results? The current implementation of Combined Results Mode automatically closes cities with zero results, and if I remember correctly, the "old" Combined Results Mode also did this flawlessly. I was going to ask if this means that SearchTempest still *does* see what you are doing (in this case with Google, where the current Combined Results come from, instead of directly with Craigslist), but I think I realize that the distinction is that SearchTempest is actually doing the direct interacting with Google, not the user... after all, the user is typing the search term into SearchTempest, and SearchTempest is doing the work of Google searching on the user's behalf. Is this a good analogy? Alternately, when using Direct Results Mode, the SearchTempest site does not actually do the searching, nor does it see the search terms input by the user; it simply provides a handy way for the user to input one search phrase sequentially and *directly* through the list of Craigslist cities. Is my understanding correct?

***EDIT*** Uh, I'm gonna save some of that Librarian analogy stuff for another post... Yikes! lol
Tempest Nathan wrote: I do understand the value for power users like yourself though, and I apologize. That said, we do have a power option that I think will work much better for you: our RSS Feeds tool. If you're looking through hundreds of cities, especially every day, this will save you boatloads of time compared to Direct Results, and shouldn't miss anything. Check it out: http://www.searchtempest.com/faq.php#rss
I've never used an RSS reader, so I suppose it is time for me to look into that :lol:

It's really late and my eyes are crossing...If you've made it this far, congratulations! Depending on whether I am understanding these concepts correctly, I hope to adjust my future feedback for the site in a way that may yet have some meaningful value.

User avatar
Tempest Nathan
Site Admin
Posts: 1371
Joined: Fri Apr 18, 2008 10:06 pm
Contact:

Re: Latest CL postings not included?

Post by Tempest Nathan » Tue Jan 28, 2014 12:23 pm

You're just about dead-on. The only exception is we can see your search terms in both modes, because as you say, they're typed into SearchTempest. What we can't see are your search results in Direct mode. We just take the search terms, do a bit of error checking on them, then use them to generate links to craigslist, pretty much as you described. The resulting craigslist page goes straight to your browser, so we don't see it at all.

The old iframe thing was the same. For a while we did try to essentially guess which cities have results, using other sources like Google, but it wasn't very reliable. And for large searches where you want direct results, the rss feeds are really superior anyway. The Direct mode is better for searching a handful of sites within driving distance. (And we have detailed instructions on our RSS page, so it should be straightforward to try, even having not used them before.)

Of course, most of the time, the default Combined mode should be easiest. As you say, we interact with Google to get the results there, so we have a lot more control over how they're presented. But there is a slight delay, and occasionally posts are missed, so the Direct mode and rss feeds have their place.

Locked