Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Previous related discussion: https://news.ycombinator.com/item?id=26328758

If things go as planned, this may become a paid, ad-free, zero-tracking search engine. I can't express how exciting this is to me.

Over the past few years, I have made several attempts to replace Google Search with DuckDuckGo. But they have all failed and I always ended up changing the default search engine back to Google. I mean, DDG worked fine for 95% of time, but the remaining 5% failure often led to some extreme frustration that I just couldn't stand. I would imagine Brave Search to have similar issues, at least in the beginning, but they did something smart to make it less painful:

> Brave Search beta is based on an independent index, the first of its kind. However, for some queries, Brave can anonymously check our search results against third-party results, and mix them on the results page.

So, if I am not satisfied with Brave's result, Google's result is on the same page, or just one click away.



> I mean, DDG worked fine for 95% of time, but the remaining 5% failure often led to some extreme frustration that I just couldn't stand.

Is 95% really not acceptable?My experience is quite different though. When I don’t get the results I hoped I just use !g. Easy. But the result are rarely any better


Honestly the main reason I ended up abandoning DDG is because you can't see the publish date on search results.

I know it's a fairly minor feature and one manipulated often by some websites, but I've still found it massively increases my chances of picking a relevant and up to date result. I didn't even realise how much I used it until I found myself getting extremely frustrated about its absence in DDG.


I used to do this, but at some point I just stopped. Google is not better than DDG. More SEO spam and much more hostile UX.

Brave has a culture of user-hostile UX too so I don’t have any big hopes for this. I like the idea of paying for a search engine, though. I would seriously consider that if DDG offered it.


> Brave has a culture of user-hostile UX

Yep. The missteps that they've made over the past few years do not give me any confidence in the future of the project.


What missteps? The only notable UX issue we've had was years ago, and was a matter of naïve design. When we were made aware of the issue, it was corrected within 48 hours. Hard to portray that as a "culture of hostile UX".


I'd say only allowing BAT withdrawal to a single hosted wallet provider that requires KYC is a pretty significant UX issue.


That’s not hostile, it’s literally a requirement in the US for crypto.


Maybe that's a sign that we should keep our browser and crypto wallets as separate entities, no?


Yea, but I mean who would want to cash out their pennies earned anyways. Only businesses/creators should be cashing out and they would need to KYC for any normal donations. Users should be just donating their pennies to creators and websites, which doesn’t take any KYC.


This was only a year ago and not a great look - https://brave.com/referral-codes-in-suggested-sites/


I haven't noticed any UX issues, great job on the browser, looking forward to the search engine. Thanks.


Maybe you could start by listening instead of accosting every comment that you find. Your incessant reply-bombing is childish and unprofessional, nobody wants to engage with someone who defends a browser like it's their sole lifeline.

Furthermore, you don't get to choose what your "UX issues" are. "UX" quite literally stands for "users experience", which is on the other side of the spectrum from "developer experience". As a dev myself, I know it's difficult not to conflate the two, but acting like issues straight up don't exist is blatantly hostile.

I have no personal qualms against Brave. I'm just another developer who wants a browser, and Brave's naive featureset doesn't appeal to me: that's fine. I'm just helping other, similar users make the right choice.


I'm responding to users. You happen to have numerous comments here which aren't entirely accurate or fair, so I have responded to you a few times. Don't take it personal; if you publish something I feel is inaccurate, I'll post a response.

Regarding user experience, I'm not just a developer of Brave, but I'm a user also ;-) Not only that, but I spend a lot of time speaking with users all across the Web, so as to understand how they're using Brave, what works, and what doesn't. I do feel uniquely qualified to talk about matter of UX when it comes to Brave.


> I'm responding to users. You happen to have numerous comments here which aren't entirely accurate or fair, so I have responded to you a few times. Don't take it personal; if you publish something I feel is inaccurate, I'll post a response.

I’m not posting this in fight mode, I sincerely hope it will help: this is user hostile.

You’re responding but you’re not listening. You’re certainly not asking. How could you be sure you know what the other people you interact with think if you feel uniquely qualified to talk about users’ experience and just brush by people who don’t feel supported in their own experience?


I don’t use Brave but I think these guys are being unfair. Your comments are generally fine because they’ve prompted responses with detail, which I as a third party prefer.

“Brave is full of UX issues” <<< “Brave allows withdrawing BAT to only a single wallet provider”.

Okay, the latter comment is way more useful to me, a lay follower than the former. And it only happens because you pushed.


> I'm responding to users.

I'm gonna have to agree with GP that you're responding too much. I don't even use Brave nor do I care but I still browse HN. Obviously different people will see things differently, but you seem very defensive and it makes you come across as difficult.

Like I said, I have no skin in this game. You are welcome to ignore what I say if you don't find it helpful.


[flagged]


I'm genuinely curious what echo chamber you think I'm in. The echo chamber where I'm unaware what people think of Brave?


Since you seem to know it all I'll just leave you be. My only actionable advice is that you should hire someone nicer to handle public relations, lest you bleed users from your own mouth.


I do believe I've been quite respectful with you. If at any time I was caustic, abrasive, or offensive, I apologize. It is certainly not my intention.


From my read through the thread, you're being very nice, but also very dismissive. That's not actually respectful, even if it's not the harsh things you describe. And it's not kind. If you disagree with someone's experience that you feel passionately about defending, you might have better luck defending it by taking a moment to think about what they experienced differently from your own experience, how much you care about that different experience, and how you might incorporate that into future action. Not everything needs public relations, and it can definitely feel uncaring if it's mostly public explainings.


People are too sensitive nowadays. Everything needs to be wrapped silk and the actual information gets shrouded by fluff.


It’s not clear who you’re saying is being too sensitive, or why you think so.


People who think it's disrespectful when a company representative do not preface every response with "I deeply apologize that you feel this way" or some equivalent nonsense. You even claim responding to something inaccurate is user hostile.


Huh? I wasn’t suggesting the person should apologize. I was suggesting they should ask about the user’s experience rather than telling it.


Or they can just tell them facts instead of pretending to care what a random hater thinks. Just like they did. They also asked what bad UX they referred to, so they did what you wanted but it was still disrespectful apparently. If genuine feedback was met with "go fuck yourself" we could maybe call it user hostility but this was not.


I feel like pretending to listen, which is what he is doing, is worse than saying "go fuck yourself" because then it would be honest. I find it bizarre you find his feedback genuine, as to me it's the same thing as "go fuck yourself" but neatly wrapped up in a "I pretend to care what you say" format.

If he came straight out and said "I don't care about your opinion and here's why" I would respect him. Instead he just talks over the people he's pretending to listen to.

His job should be to solicit feedback, not dictate it. And if he's going to dictate it I wish he would at least be upfront about it.


You put it better than I could have before I got back to this. It doesn’t really count as “asking” if you’re preemptively dismissing the answer in the same comment.


I respect the candid comments from the Brave Dev. I wish all "PR departments" acted like this to be honest.


I see you put the twit in twitter.


We've banned this account for repeatedly breaking the site guidelines.

If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.


Why not try Neeva? They are going the route of a paid search engine.


Upvoted, but here is the reason why I don't use it, some people haven't yet fully realized that Internet doesn't have borders:

> We will be in touch when we are ready to release Neeva in your country. Thank you for being part of the Neeva team, we are so excited to build the future of search with you.


I’m using Neeva. I like the team and the idea, but at least for me there is a drastic drop off in search quality from google. It is pretty far from 95% as good.


Can you elaborate on how you feel "Brave has a culture of user-hostile UX"? You're not talking about the first version of the User Tipping feature from 2018 (where Brave gave BAT to its users and asked them to give mark which creator(s) they'd like to support) are you?


Generally the same type of problems as a lot of UX has today, especially on mobile: various messages and modals and controls that seem to be motivated by Brave’s needs, not mine. Sponsored images, trying to get me to set it as standard browser, “Brave rewards” whatever that is being a permanent part of the UI and turning itself on without me asking it to.

These might be small things compared to Google, but I’ve never experienced that DuckDuckGo did anything like it, so my trust in them is higher.


Let me expand a little on why I think this is so corrosive to my trust in Brave, because this is interesting stuff. When I use the Brave browser, I have to second-guess everything in the UI to consider why a control or message is there, if it’s in my interest or if you’re trying to get me to do something that’s in your interest. My eyes have to scan the UI in much the same way I do with ads in search results or spam in my inbox; having to actively filter out the potential harms from the things that are useful.

It’s like I can feel my eyes getting more tense as I do this.

That means that every single time I use the browser, the impression that Brave should not be trusted is reinforced in a very physical way. It’s not just a “brand impression” but a muscle memory.


Yep, agreed. I ended up switching to Firefox as a result of things like this, which was good in many ways but took a lot more configuration.


After using the duck for a couple of years, I have become better at two things:

- Reading man pages or official documentation sites before opening a search engine

- Thinking of more precise search keywords, as I got used to duck not helping me as much as google


Along these lines I use ddg’s bangs for the same benefit. So many searches for Python help are filled with very shallow intros on tutorial sites of varying quality with the official docs rarely the first result.

Now I just prefix my query with !py and I’m immediately taken to the docs.


Brave Search supports !bangs ;)


That’s good to know!

For clarity, I wasn’t trying to say DDG is better than Brave, rather agreeing with the parent that there are smarter tools for gathering information rather than relying solely on a search engine.


Indeed, including nearly all of DDG's !bangs :) We also add in some others, such as !so for StackOverflow, !gh for GitHub, and !mdn for the Mozilla Developer Network.


Those are also in DDG.


Wouldn't surprise me; DDG has an impressive list of !bangs (more than 13K, IIRC). Thanks for the clarification!


And is there any list of all available bangs + also do we have an ability to add new ones?


Tip it lacks keyboard shortcuts to navigate search results (up/down, enter).


Arrow and PgUp/Dn keys scroll the page, as per standard Web UI. Tab key jumps from one link to the next, also as per standard Web UI. Maybe they changed that between our comments.


That works horribly on the results page, around 10 tabs to get to the first result.


also: considering how far out in the long tail of search terms my query is, before choosing to go with the !g bang out of the gate.

Google I find is still better for topics that are more idiosyncratic. But the bang syntax makes DDG a natural choice as default because many times I'll want to go directly to a specific domain search, e.g. !r or !nyt


> Is 95% really not acceptable?

No: When you use web search for professional work, such as searching for error messages or description of bugs of some software, any miss of somebod having encountered and solved them before can cost you days of work.

From my experience, Google is currently still the best at finding those.


I think before you spend days of work on something DDG can't find, taking a few seconds to add !g and check Google's results would be sensible. Usually when I try that, though, Google isn't any better.


I use DDG on personal devices and Google at work. I have a work-issued Google account, so privacy isn't really tractable for work stuff anyway


> When I don’t get the results I hoped I just use !g. Easy. But the result are rarely any better

This is exactly my experience. I have a "failed" search probably about a quarter of the time. Changing around the keywords can sometimes fix those failures... maybe about a quarter again are still stuck. So, yeah, ~5% failure rate. I inevitably try !g and am inevitably disappointed with effectively the same results (or lack thereof). Google successfully recovers a failed search maybe 10% of the time.


> Is 95% really not acceptable?

Moreover, is any search engine really at 95% success rate? I certainly have never gotten that high with Google, even back in the days when Google Search was good. Nowadays it's like 85% or so. About the same as DuckDuckGo for me. No matter which one I made my default, I'd have to check the other occasionally. (Incidentally, the same is true of satellite imagery. Sometimes Bing Maps is just much better for no obvious reason.)


It isn't acceptable, no. I tried Duck Search (aka Bing) for a couple weeks and in the beginning I wouldn't know that I wasn't getting the results I was looking for and eventually realized that the results just sucked compared to Google.

I found myself having to second guess the results and then did a Duck / Google hybrid for a while, going to Google when I didn't get what I was looking for and eventually it was too much friction. I equate it with when I used to use two different text editors, one for speed (Sublime) and another (IntelliJ)for step-debugging because Sublime didn't have that part well implemented and it was just maddening to have to switch back and forth all the time and learn/maintain two sets of keyboard shortcuts etc.


> When I don’t get the results I hoped I just use !g

I use !sp instead, same results and no Google tracking

(!sp searches on Startpage which in turn uses results from Google; according to both Privacy Badger and Brave Shields there are no trackers on SP)


I sometimes wonder if that 5% is something DDG and others can realistically solve. Perhaps the issue has less to do with engineering and more to do with Google being the dominant player over the previous 20-years (give or take). That's an awfully long time for one company to effectively own a product category and build expectations among users about how it should work.

FWIW I do get good results from DDG (sometimes better than Google) but that does require me to be a bit more thoughtful with my queries.


For me it's random technical dumb stuff, like library version compatibility. Or a specific syntax I know exist but I can't figure out.

Now when I don't find what I need, I double check with g! ... once every 2 or 3 times, google do find what I'm vaguely remember exist and is out there.

Is never actual content, it's when I look for a specific one liner to copy paste and DDG do not deliver.

I can live with that.


I am on the third or forth trial to change to DDG and this time it is working not because DDG is better but because Google's search is degrading so much.


I tried brave search today and the results were rather good. I have no idea what 95% working means but this is a nice start


In my experience, without use of g! DDG isn't serviceable.


DDG already includes Google (amongst others) in its results. It's not just a front end to Bing, like many assume (though I can't find DDG's article on the subject to directly cite).

The thing with people who switch to DDG is, they do so consciously for privacy reasons but then forget that the reason G's results are so good is because they add little bits of context in through their profiling. But that doesn't mean that DDG's results can't be as good as G's, it just means you need to add that context yourself. Like if you search for a coding problem, add your computer language name to the search query. Or if you're search for a restaurant, add in your home town too.

I've found DDG's results to be comparable to Google's and in fact in the last ~3 years of exclusively using DDG, I can count on one hand the number of times I've tried searching for something in Google after a failed search in DDG.

Image searching is a little more hit and miss though. G's image search is better -- generally speaking. However DDG doesn't include Pinterest spam. So if you're after something specific using image searching, Google is better. But if you're after a general list of usable images, then DDG is better.


> DDG already includes Google (amongst others) in its results. It's not just a front end to Bing, like many assume (though I can't find DDG's article on the subject to directly cite).

Per DDG's own help pages, they use mostly Bing and no Google. (The mixture of over 400 sources that they claim is used to provide the infobox-type results, which they call "Instant Answers", not the regular search results)

> We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).

https://help.duckduckgo.com/results/sources/


> but then forget that the reason G's results are so good is because they add little bits of context in through their profiling

I disagree. I've tried to switch to DDG several times too, and always go back to StartPage, which is a proxy for Google, and the results were always better even without profiling. There is a case to be made for Google having better results globally from aggregate user behaviour though.


I think googles crawler excels at crawling forums and social media like stack overflow or Reddit or support forums. The main thing I use google for is when I have a bug whose solution is 9 pages into some thread on some obscure discussion board I’ve never heard of


G's results are terrible for over 5 years and now they are reaching uselessness levels of Altavista.

No G, I didn't want to search for what you suggested, really not.


> "This new-fangled technology is worse than what we had back in my day!"


DDG also makes it very to download images, unlike Google that now makes you go though to whatever site and have to scroll around in the page to find the image


Google was forced to do that because sites where screaming google is stealing their traffic by linking directly to images.

Once DDG is big enough, they will be forced to do the same.


Or as people like to say about Uber, DDG is doing "legal arbitrage" since they're still small and Google can't do that anymore.


The best thing about Google is that if you're on a website - say a travel website looking at hotels in Thailand - and then you Google "USD" it automatically completes "USD to Thai Baht" and then just gives you the answer very user friendly automatically. The same is true if you'd search "Best p" - you get "Best places to visit in Thailand" immediately, with a bunch of cards that are easy to read and use and get to more relevant things you're looking for.

Sure - this invades your privacy. But it leads to good results and a better experience.

You really don't want to give up your privacy in exchange for nothing. But this doesn't feel like the case here.

It's also just as good for location things. If I'm in different neighborhoods - I can type one letter in - and Google will pop up the right restaurant - and then in one or two clicks I can make an order.

This feels like a trade I can live with. I get it that a lot of people can't.


You're happy to give up your privacy just to save typing a few characters?


Considering that I use Google instead of DDG primarily because of how good the contextually aware results are for the things I most commonly search - sure.


> Or if you're search for a restaurant, add in your home town too.

This definitely doesn't help in India. If you live in the US, Brave/DDG can probably serve you better local results. It's abysmal in India. Google local results in India are orders of magnitude better here.


I live in Europe so I'm definitely not talking from a US perspective. But I acknowledge that web services are generally worse in India than most (other) developed countries.


This exhibits the halting problem, no?

I tried to use DDG, but now I don’t because I never knew whether it was just my DDG search that had been unsuccessful or whether there were truly no results.

In fact, I wouldn’t know with any search engine whether there are “truly no results”, so I use G because I prefer to get what’s widely accepted to be the closest results possible.


DDG almost fails in the same way as goog.

The only advantage I have seen in results is Google has removed nonsense and conspiracy garbage from their results.

It's the general rush towards infantilism. I want to see the kind of nonsense that has assuaged so many people into effective insanity and apparently Google has decided that I'm not enough of an adult to view it.

It's like their Android dictionary. It lacks lots of words that I have to go and check manually because they've decided that someone using a nuanced word is only done in error

Or with their search where they remove all the important stuff from the query and return the results that I was specifically trying to avoid with those important modifiers "there's not many results with x" - Yes! That's the point.

All over the place they're just on some endless campaign to patronize the userbase. From the address bar that simply just refuses to do http: to a painfully dumb search in Gmail, it's really a company wide systemic problem they need to address.

At least back in the say AOL days, which were renown for this kind of mentality, they kept things at a stable sophistication for the lifetime of the product. It wasn't in some unending rush to become ever more stupid and childish with every subsequent release


Same excitement here. I'd love to see anyone(Brave or others) chip away at my Google dependencies, even if they charge me for them. I already love Brave as a browser, so here's hoping search pans out.

I don't mind a company profiling me. A lot of Google's cross interacting products work great(Gmail to Calendar and Maps, for example). I just don't want them advertising to me, or selling my data. A guy can dream...


> I don't mind a company profiling me

Not everyone has that luxury.

> I just don't want them advertising to me, or selling my data

Then your data is worthless to them. Nobody wants companies to abuse their personal data, that's why it's such a lucrative business. Companies like Apple and Brave get away with it by edging out competition and instating their own standards (see: Brave's Ad "replacement"). It's all so ridiculously asinine that it makes me want to uninstall every piece of software from my computer and use it exclusively as a space heater for the rest of my life.


> Then your data is worthless to them

This is precisely the point. It should be worse than worthless, it should cost them money. And they should charge that to me, plus some nominal fee. I guess that's what I'm asking for.


The issue is that you're not describing a sustainable business model. Sure, it would be a much better situation than we have now, but you can't cover hosting fees with data bills.


That's a strange argument. "If we pay you for the product you give us, we'll never be able to pay our service providers. Instead, we'll give sell it to someone else."


Uh oh, someone better tell Microsoft that charging businesses for Exchange isn’t a sustainable business model.


If there was a broadly enforced requirement to pay users for their data, businesses would adapt.


How much money does Google make again?


Brave isn't capturing user data, and the "replacement" topic requires more nuanced coverage. Internet users have been installing ad-and-content blockers long before Brave (when Netscape launched the Plugin API back in 1996 or so, ad-blockers began to appear almost immediately).

Brave is rescuing the Web from a block-alone response, which starves content creators of much-needed support. Brave also increases the potential for support by giving users without disposable income (and those with disposable income) the ability to support those who make the Web enjoyable. We do this in a manner which is low-friction, and anonymous too (thanks to the Basic Attention Token).

Brave has introduced a model that understands the security and privacy reasons for blocking third-party ads and trackers. But Brave doesn't stop there (as is the case with popular blockers); it also aims to address the issue of content sustainability online.


You should read The Age of Surveillance Capitalism.

https://m.youtube.com/watch?v=swMo1sK5ntk


> > I don't mind a company profiling me

> Not everyone has that luxury.

Defend that statement.


> I just don't want them advertising to me, or selling my data.

Google doesn't sell your data.

They sell the opportunity for companies to place their ads based on Google's placement algorithms, which use your data.

There is a big difference, and while I don't expect the general population makes the distinction I think on HN people should understand this.


> I just don't want them advertising to me, or selling my data.

You can get pretty close to that by buying YT Premium [if you watch YouTube] and using an adblocker everywhere else, and this gives Google the non-ad-based monetary incentive to profile your viewing habits to show you more videos it thinks you'll like without optimizing for Ads.


I've stuck with DDG for about the past year and a half and I have to agree. I want it to be good, but when debugging some obscure problem (like trying to learn SwiftUI lately) Google is able to dig up more result. Of course the "there were not results for <your error message> so displaying results for 'computer programming instead'" is frustrating so I preemtively add quotes around every term more frequently.

Lately I've noticed a weird problem with DDG where it will load a blank page of results. The page header with logo and search bar is there, but the white part of the page that has results never loads. Even after refreshing multiple times it's the same, but trying a different query fixes it.

If Brave can manage to produce higher quality results while weeding out SEO spam I will definitely subscribe.


> a weird problem with DDG where it will load a blank page of results

I've noticed the same problem too. My guess is that DDG pulls search results from 3rd-party engines such as Bing and for some technical issues it may fail from time to time.


I wish they would add a feature that would let you add your own tags to counterweight the lack of tracking.

Right now google knows a lot about you and uses it to refine the search results, if you remove tracking - quality drops. But if you at least let the user tell the engine that "I'm a programmer, gamer, geek, whatever" it might just do the trick to counterweight that.


Hey, we are planning on implementing something very similar to what you describe. You can read more about our proposal here: https://brave.com/goggles

In a nutshell, community-curated lists of rules to deeply change the way our core ranking algorithm surface content and influence the results you see for a given query.


DuckDuckGo allows you to do a Google search by prepending "!g" to any query. So usually I do that for the last 5% of queries that DDG fails on.


You can use !g on Brave Search too, or turn on Fallback Mixing in Brave Search Settings, which will anonymously call out to Google and pull in results as needed. This helps to train the nascent engine more rapidly. I hope this helps!


I am not able to locate this setting. I am using Brave Search on Firefox. Is that available only on Brave Search on Brave browser?


Yes, the Fallback Mixing requires the Brave browser, since it pipes the request through the participating user's machine (only if the user has first opted-in to the feature).


Is that legal?


Why wouldn't it be? Google scrapes the web to populate it's results, why wouldn't other search engines scrape the web as well? Google is a website.


It's not scraping static text in order to point you to those sites, it's using the features of the site to perform a service better than you can do yourself. It's completely different.

If I made a site that claimed to help you with your math homework and simply sent the queries to WolframAlpha, that would also not just be "scraping."


This is basically what Google does to Wikipedia and rebrands as "Knowledge Graph".


Google has donated many millions to the Wikimedia Foundation, basically in payment for this.

(But, again, there's a difference between scraping and echoing a request on another site and waiting for its response. The latter is basically unauthorized use of its API, not scraping.)


It'll be very telling if Google switches to using Wikipedia Enterprise: https://www.wired.com/story/wikipedia-finally-asking-big-tec...

My guess is Google's "donation" is pennies on the dollar from what they benefit from Wikipedia, and more of a token gesture than anything else.


This is naive. Web sites have various terms of service.


Violating a website's ToS is hardly illegal, though.


> Violating a website's ToS is hardly illegal, though.

Can you be more specific as to your claim? Not illegal in what sense(s)? And what is your basis for the claim?

I'm not a lawyer, but saying "violating a website's ToS is hardly illegal" is fraught advice.

While individuals may get some leeway when it comes to ToS violations (see [1] and [2]), I would expect companies scraping and/or extracting content would be treated differently.

[1]: https://www.eff.org/deeplinks/2010/07/court-violating-terms-...

[2]: https://arstechnica.com/tech-policy/2020/03/court-violating-...

[3]: https://www.octoparse.com/blog/10-myths-about-web-scraping

[4]: https://www.law.com/newyorklawjournal/almID/1202610687621/?s...


The only way for it to be illegal is if they're breaking a law, but then it would be illegal regardless of what the ToS says.


Because Google's robots.txt disallows it, and those websites allow it.


robots.txt is not a legal contract. It's just a convention to express the wishes of the site author, but there's no legal obligation to follow these wishes.


It does indicate that those other sites want Google to scrape them, while Google does not want others to scrape their results, which is an important distinction ocdtrekkie ignored for whether the scrapee will want to take legal action.


You may wish to review https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-l...

Google Search results are definitely "public data" so long as Google provides them to anyone who asks.


Then why does Startpage pay Google and DDG pay Microsoft?


While scraping search results isn't illegal, by any means, it's also not illegal for Google or Microsoft to block requests they believe are from competing search engines. Presumably the cost of paying them is less than the cost of hiring engineers to constantly try to find new ways to outwit Google and Microsoft engineers.

Again, if scraping data from websites without permission, Google simply wouldn't exist. Bear in mind, robots.txt is a feature that Google and Microsoft choose to respect, but the default assumption search engines have made from the beginning, is that they are free to grab whatever they want from the web, unless you ask them otherwise to please not.


> the default assumption search engines have made from the beginning, is that they are free to grab whatever they want from the web, unless you ask them otherwise to please not.

Which Google's robots.txt does.

> scraping search results isn't illegal, by any means

While scraping the results for yourself to look at might be OK, scraping results to display verbatim in another search engine without permission stretches fair use.


> While scraping the results for yourself to look at might be OK, scraping results to display verbatim in another search engine without permission stretches fair use.

No, it doesn't, because Google results aren't copyrightable, hence, there is no such thing as fair use. It's just information anyone is free to collect and use as they see fit.


Why would rankings not be copyrightable?


Why would they be? Again, if all things being copyrightable by default, Google could not even exist, they assume they have the right to consume any data they want.

If a monkey can't copyright a selfie because they're not a person, an algorithmically generated spew of stuff Google ripped from elsewhere certainly lacks merit for copyright.


All things are copyrighted by default. Once again, those websites grant a license to search engines to consume their content via robots.txt, and Google does not.


It's just a redirect to Google.


There seem to be two different features:

1. Redirection to Google.

2. Piping Google's results back to Brave Search's UI: https://news.ycombinator.com/item?id=27594754

The original commenter was asking about the latter.


Ah, I see, Thanks for the heads up.


Is scraping Google for your own commercial search engine legal?


The frustration I am talking about is:

Check query result -> realize the result is bad -> scroll back to the search bar -> place cursor at the beginning of the query -> enter "!g" -> redirect to Google.com

It is not too bad on desktop, but doing it once on a smartphone is more than enough to push me to switch back to Google.


The !g can be almost anywhere in the query, it just can't be followed by non-whitespace or prefixed by special signs.


This is what I wanted to say. It's a life saver on mobile especially. Just append !g to the query (or !gi for google images, !gm for maps, etc)


Personally I find it pretty predictable which queries duck duck go will fail on. Basically very niche ones.

So I just prepend based on what I'm searching for to begin with instead of after a failure. But I've also been using duckduckgo as my default for over a decade so I've gotten used to it.


If you use Tridactyl, you can quickly jump to the search bar with “gi”. That's what I do.


you can easily create a command for it (and bind to key(s)):

  command ddgGoogleBang composite js (new URLSearchParams(window.location.search)).get('q') + ' !g' | urlmodify_js -q q | open


I'm amazed that DDG/Brave don't understand that typing something immediately after a "!" on a smartphone is annoying and difficult. DDG only supports "!" after the character for a small subset of bangs. Thankfully "g!" is one of them, but it's immensely easier to type on smartphone.


most browsers support <c-l> to hop the cursor to the url bar for searches or url entries


    / <c-e> !g <enter>


Or you could just use https://startpage.com/ and get Google quality results 100% of the time.


that's also 100% reliant on google though. what would happen if that became a lot more popular than it is now? would google try to sabotage it in any way?

i would rather support and spread the word about search engines that don't rely solely on google


> that's also 100% reliant on google though. what would happen if that became a lot more popular than it is now? would google try to sabotage it in any way?

Yes, same applies to DDG/Bing though.


It redirects to Google. So the effect (including tracking) is identical to just doing a Google search.


This is incorrect. Fallback-Mixing, if you have enabled it (which requires Brave), issues an anonymous query to Google, lacking any cookies or other persistent state for that domain. These results are then presented along with Brave Search's own results. There's no tracking involved. If you perform a direct Google search, you're passing along your cookies as well.


The user was replying to using DDG with !g

In an indirect way, it’s possible by doing !sp on DDG, which redirects the search to startpage, which shows untracked google results.


Ah, my apologies for any confusion. For what it's worth, the !sp bang is supported on Brave Search as well :)


As I said in my other comment[1], you might be interested in using !sp which gives you the same results without Google tracking

[1] https://news.ycombinator.com/item?id=27598042


Startpage is owned by an adtech company now.


that doesn't change the fact that there are no trackers and the fact that it uses Google results


No, but it does mean that you are likely to use it without knowing when they do start tracking you. Because of course an adtech company will (and of course they will deny until the day they change it).


> Because of course an adtech company will

then it seems they're being quite slow, since the acquisition happened almost 2 years ago.


This is silly. 95% of what one can track on the front end can be done on the back end. If you hit Google's servers, you're being tracked. Period.


They will definitely implement ads. They're an advertising company.


"options for ad-free paid search and ad-supported search"

https://brave.com/brave-search-beta/


Nice, I would pay for it. Would make it a lot easier if the backend was #opensource though. Not saying I won't but it would make it a no-brainer.


Future post: "We have listened to our users, and we are removing ad-free paid search due to a lack of demand and [some excuses about how it's technically difficult to maintain it]."

We'll see which comes first. That post, or "Our Great Journey."


Just curious, why are you opposed to search ads? They are already targeted by your search query, so don't fundamentally rely on tracking data.


Since they're nearly indistinguishable from real organic search results, people click on them assuming the search engine found them the best result. This leads to two major problems:

1. Search ads are the primary source of malware and fraud on the Internet today. (Phishing emails are second.) Sites pretend to be other sites all the time, and to allow tracking and landing page behaviors, every major search ad provider allows ads to "lie" about the destination domain. So you may see an Amazon ad, it says it goes to Amazon.com, but actually directs through to realamazonlinkipromise.biz instead. Fraud's really profitable, so fraudsters win ad slots easily, and are adtech companies' best customers, so there's really little incentive to crack down on this.

2. Search ads use this placement as a form of extortion. If you run, say, Best Buy, you shouldn't have to buy search ads for "best buy", because obviously you're the best result. However, they have to, because if they don't, the search engine will sell ads to their competitors using their keyword, so people searching "best buy" get "Circuit City" as the top result instead. (Yes, I chose that reference in part because I don't want to shame any real current companies in this example for sleezy practices.) And since users click the top result (the ad), not the first organic result, Best Buy ends up paying for every click for every user who goes through Google/Bing/etc. to get to Best Buy.

The second reason is why browsers are so obsessed with combining the search and address bars: They want you to search "best buy" or "bestbuy" or etc. because that's ad revenue, whereas actually typing bestbuy.com nets Google nothing.


> 2. Search ads use this placement as a form of extortion. If you run, say, Best Buy, you shouldn't have to buy search ads for "best buy", because obviously you're the best result. However, they have to, because if they don't, the search engine will sell ads to their competitors using their keyword, so people searching "best buy" get "Circuit City" as the top result instead. (Yes, I chose that reference in part because I don't want to shame any real current companies in this example for sleezy practices.) And since users click the top result (the ad), not the first organic result, Best Buy ends up paying for every click for every user who goes through Google/Bing/etc. to get to Best Buy.

One thing to note is that the cost of the ad is based on the landing page relevance (and even more so for branded terms), and so in your example Best Buy would be able to buy the ad for the "Best Buy" keyword for pennies (a rounding error on their SEM campaign, I'm sure), while Circuit City would have to pay a whole bunch for the "Best Buy" keyword.

Given that, I don't mind it so much. It's a good way for a competitor to get their name out there, but it's not really a sustainable practice long term for them. There's built-in pressure favoring the incumbent on their own terms.


Bear in mind, Best Buy should pay zero pennies for each of the millions of people trying to reach their website. It's absolutely inexcusable for a search company, which also owns an ad company, and also happens to run the web browser everyone's using, to create a system that basically taxes all attempts to visit a business's website specifically.

Honestly, what Google and Microsoft and such are doing in this case is trademark theft. They are selling the search result for a trademarked name they don't own, and when the actual trademark owner wants to be found by their own name... they have to pay for it.

I don't know who is going to file the case, but sooner or later, someone should, because it's a slam dunk.


> every major search ad provider allows ads to "lie" about the destination domain.

This seems like the real problem, not search ads as a concept

> So you may see an Amazon ad, it says it goes to Amazon.com, but actually directs through to realamazonlinkipromise.biz

Just disallow that? Problem solved.

If you want yet another filter, only allow public companies or companies that have raised >10M on Crunchbase to advertise, and have them verify that they are really who they are by asking them to put some string of your choosing in their DNS records.


> Just disallow that?

Sure, Google could disclose real advertisement destination URLs tomorrow if they wanted. But the marketers are their customers, and that would upset their customers quite a bit. Especially since a lot of their customers' entire purpose in paying for Google Ads is to exploit that particular feature.


Google only charges users once per user per click.


Honestly, ads clearly marked as ads and contained just to the results page would be fine.

But in the race for better ad performance, they introduced tracking & retargeting & profile building while at the same time both minimizing the visual difference between ads & organic as well as nearly pushing organic of the first page.


IMHO search ads are bad search results, and in 99% of time they are not something I would want to click. So they are doing nothing useful but only adding friction to my search experience. I am not against privacy friendly ads in free services, but if there's an option to pay to get rid of ads, I will pay.


I'm in eCommerce, run ads (on eBay, not Google) and would have to agree 100%.

The only products we put paid promotion on are common and overpriced, and the demographic of the customers is very different to what we'd find through our products sold through organic search.

Based solely on anecdata through the messages we receive and the addresses we ship to, the people who click on ads are somewhat more likely to be lower-socio, much more likely to have low literacy skills, and a couple of orders of magnitude more likely (not an exaggeration) to live in a remote Aboriginal community.

It feels somewhat dirty/exploitative, but it's what the customer wants. They have the choice of saving $50+ by scrolling past the first 3-4 results, but they choose not to. I just don't understand.


I can't answer him, but there is a peace of mind not having ads talk (is not the right word) to you, even absent everything else. As in I can actually set my attention to something and finish a coherent whole without having to give any attention to ads.

I didn't realize this until I installed Sponsorblock which got rid of the last ads I was seeing there. Suddenly I could focus on whatever they were creating and the video was talking about without having my attention diverted to something else.

Maybe that is just me, but that is why I now mind ads.


Since the brave search results are so thin, maybe put them on the right to be less confusing.


Search ads have become visually almost indistinguishable from legitimate search results on most search engines, including DuckDuckGo. That's why I try to avoid them whenever possible. DuckDuckGo allows you to turn them off completely.


> Brave Search beta is based on an independent index, the first of its kind.

That's... an interesting way to put it. I can't really twist and turn it into the truth though. Smells like it was put through a lot of PR.

>However, for some queries, Brave can anonymously check our search results against third-party results, and mix them on the results page.

This is something almost all of the search engines outside Google and Bing does. DDG does this with Bing for example.

As far as I can tell the only "new" in this will be that the same thing is done again by another company.


"If things go as planned, this may become a paid, ad-free, zero-tracking search engine."

In 1998 when Google's founders announced their search engine they claimed it would be less commercial, more academic, more transparent and they would avoid the influence of advertising. Did things "go as planned." Not even close. What is the lesson here.

Meanwhile, every search submitted during the "beta" period is subject to none of those limitations.

"So, if I am not satisfied with Brave's result, Google's result is on the same page, or just one click away."

Brave is not the first to do that. Check out Gigablast, for example. If I am not mistaken, they also claim to use an "independent index". At least, they provide the source to a crawler and server.^1 That is what people should be excited about. Not Gigablast per se but the idea of an open source search engine that anyone can run. searx is another project worth looking at.^2

How many ways are results promoted and demoted; what are the factors used. Are these search engines that comenters are recommending in this thread transparent. (Making promises on a blog is not "transparency" IMO.) Where is the source code. What are the various server settings that alternatives like Google, DDG, Brave, Startpage, etc. never provide to users. This stuff should matter, yet the discussion of search engines always seems to devolve into personal usage anecdotes and "search shortcuts". Every user has different needs and preferences.

There are many knobs in web search that advertising-supporting tech companies providing "search engine" websites will never let users twiddle. The source code for those servers is not public.

1. To get an idea of type of settings users of popular search engines are not being allowed to control:

(Scroll down to "SEARCH CONTROLS") https://raw.githubusercontent.com/gigablast/open-source-sear...

2. https://github.com/searx/searx

For a current list of searx public instances

   curl https://searx.space/data/instances.json|grep -Eo '(A\+", .[^"]*.{4}[^"]*)'|cut -d/ -f5|uniq|sed 's>.*>https://&>'


hi, gigablast creator, matt, here. thanks for mentioning gigablast. i've been coding web search engines for almost 25 years so it's always nice to see ppl recognize. i wish more ppl would care about these things. with enough people caring i think i (or we) could make gigablast into a super transparent, private search engine that doesn't rely on big tech like the other guys. really i just need more hardware at this point as that is the main technical hurdle for improving results quality and performance. if somebody would give me like $1M in amd-based minicomputers (i like minicomputers better than big servers - preferably asus) i think we could have something much better and faster, although what is there is pretty good -- this might be enough to really get things going.


in the distribution available on github.com, included are some static binaries for various open source programs. any reason that a static gb binary could not be distributed as well. with linux, i use musl so prefer static to ones linked against gblic.


anyway, thx for sharing some of your work. very much appreciated.


> searx

I just gave the docker image a try. It works out of the box but the search time feels slightly more than that of Google - which is one of the reasons I gave up on DDG.

Also, I am curious - if I am hosting it on my own server and using Google as one of the engines - does that not mean my search ultimately goes to Google and they can still profile me?


I just tested the 47 servers listed in https://searx.space/data/instances.json. I did not use a browser. No Javascript, cookies, etc.. A good number of them worked fine.

Who knows what the people running those instances do with the search data they acquire.

What I like about searx though is the list of search engines it potentially targets. Comprehensive lists of search engines on the internet are always valuable. I see searx as a supply of "parts" with which one can make something of their own. I have made a metasearch utility for myself.


What does your metasearch approach look like?


A. Text-only

B. Search from command line

C. Can open an index.html of saved search results in any browser; each query gets its own SERP; search results are saved in a directory that can be tarballed and compressed allowing simple transfer to any computer with a UNIX userland

D. Easy to add new sites; follows a failry standard template; currently at only eight sites, but adding more (like the ones in searx)

E. Requires only standard UNIX utilities; consists of small shell scripts of less than 2000 chars

F. Fast; no cruft

Unique features:

1. Streamlined SERP; URLs only, minimal HTML, i.e., <a>, <pre>, <ol>, <li>, <!-- -->; no images, Javascript or CSS; SERP contains timestamps in HTML comments to indicate when each query was submitted

2. Each SERP contains deduped batches of results from different search engines; source search engine indicated by short prefix; if desired, can resort to intersperse results from different sources, e.g., sort by URL

3. Continuation of search; allows retrieval 100s of results by spreading searches across periods of time too long for websites to track, thus allowing retrieval of large numbers of search results while avoiding ridiculously small result limits or temporary bans for "searching too fast" <--- I could not find anyone else using this approach

4. By default only minimum headers sent; custom headers can be sent when appropriate for particular site, e.g., DNT to findx.com; allows for complete customisation of presence/absence/content/order/case of HTTP headers, thus can potentially emulate any browser or other HTTP client (also supports HTTP/1.1 pipelining which curl cannot do)

5. Can be used with any TCP client; not limited to one library, e.g., libcurl; works great with proxies like stunnel and haproxy

6. URL params or hidden form fields that can potentially be used to link one SERP with another SERP are removed or rendered ineffective


This Brave search beta is quite slow and there is no way to access "page 2". One page of results. Hard to believe this is "good enough" in 2021.


Hey, Brave Search is currently hosted in the US (I am assuming you are on another country) and latencies will be bigger in other places. We will scale in the future, which should improve speed.

Regarding number of pages of results, please see my answer on another thread: https://news.ycombinator.com/item?id=27597911

TL;DR: we currently show the equivalent of 2 pages of results and will consider more in the future.


> I mean, DDG worked fine for 95% of time, but the remaining 5% failure often led to some extreme frustration that I just couldn't stand.

So, just use https://startpage.com/ and get proxied Google results. Searx is another alternative.


You can't have something be zero tracking and paid since they need to know if you have paid and so need to be able to track you.

I use DDG for my main search, but there is the !g (i think) that you can prefix a search with to get it sent to google through DDG.


Tracking isn't a necessary component of a subscription service. You can have a model with premium features and/or offerings which doesn't harvest user data, such as searches and more.


I guess you could, but then I would have to depend on the service to hold up their promise not to track me. I don't want that.


> I have made several attempts to replace Google Search with DuckDuckGo. But they have all failed and I always ended up changing the default search engine back to Google.

Hmm...

> Brave Search beta is based on an independent index, the first of its kind. However, for some queries, Brave can anonymously check our search results against third-party results, and mix them on the results page.

That's also what DDG does. If you don't like DDG, odds you'll like some even smaller effort are quite to zero.


Do they anonymously check Google though? Maybe they'll just use bing like all the others


I switched to using StartPage after DDG failed me. I ended up adding !g to almost all of the searches.

StartPage shows Google results through a proxy for improved privacy. I am quite happy with it.


Just a note that StartPage was bought by a not-too-privacy friendly company.


Startpage person here. In 2019, Startpage announced an investment by System1 through Privacy One Group, a wholly-owned subsidiary of System1. It's 2021 and our privacy policy hasn't changed.

What has happened with this investment, we've hired additional engineers and added new features.

And, System1 doesn't receive any user personal data because we don't collect it and never will. Why did System1 invest? "System1 is interested in Startpage's ad revenue, not its data" Source: https://www.computing.co.uk/news/4017337/privacy-focused-sea...


Startpage was turned into shit over the last few months. Requiring JS for search and absolutely freakish amount of ads dominating the first scroll of results.

Been with them for about a decade but this is too much


Hi - I'm a person from Startpage. I'll share your feedback with the team, but here I can respond to some of these things.

JS: Startpage does use Javascript, but only to enhance user interaction with our site and improve the search results we provide.

ADs: Startpage displays at most 3 ads. Like other private search engines, Startpage's revenue model is based on contextual ads.

Let me know if you have questions or suggestions.


For the other 5% of the time, use the !s bang for Startpage. Anonymized Google search results, with an option to visit the websites on the results page anonymously.


If DDG works in 95% cases, just use it and use !google command on it when it misses the mark.


or even better you can use !sp

https://news.ycombinator.com/item?id=27598042


Startpage got sold


I'm aware, that was almost 2 years ago and startpage still has no trackers (as of today, according to EFF Privacy Badger), so I consider it safe.


Fyi. In Firefox you prefix your search with the engine you want to use. That is, DDG can be your start and you can conveniently use Google as needed.


so you're saying it would be better then DDG just because u r paying money for it?


> Over the past few years, I have made several attempts to replace Google Search with DuckDuckGo

I have been using Startpage for a while. It's results are same as Google, but with zero tracking. But it puts (non-personalized) ads in results.


DuckDuckGo does the exact same thing with shebangs. Brave's implementation is just less granular.


Brave Search supports the Fallback Mixing option, as well as shebangs (e.g. !g, !a, !b, !d, !e, !yt).


They are called bangs, not shebangs. A shebang is #!

> In computing, a shebang is the character sequence consisting of the characters number sign and exclamation mark (#!) at the beginning of a script. It is also called sha-bang,[1][2] hashbang,[3][4] pound-bang,[5][6] or hash-pling.[7] - Wikipedia

> Bangs are shortcuts that quickly take you to search results on other sites. For example, when you know you want to search on another site like Wikipedia or Amazon, our bangs get you there fastest. A search for !w filter bubble will take you directly to Wikipedia. -DDG


Thank you for the correction. I used to call them hash-bangs (not sure where I picked up that habit in my ~25 years of industry experience), until I started seeing more people refer to them as shebangs. Oddly enough, while reading them out I would always say "bang, <identifier>". I'll try to refer to them simply as bangs (or perhaps something like search/filter bangs) from now on


Because '#!' is a hash followed by a bang.


Quite obvious, but naming things is funny in the web-development world. We still call any asynchronous retrieval of data "AJAX," even though it rarely, if ever, involves XML :-)


> Brave can anonymously check our search results against third-party results

Why are we OK with these free services literally stealing Google results? Could you imagine the backlash if it was discovered that Google was doing the same for its results by stealing from DDG or another, smaller player?


Google is a Search Engine. Further, it displays content from sites directly in its results. This includes recipes, show times, sporting event details, and more. It has been argued that Google is stealing this data from smaller sites. Brave is (optionally, if you enable the feature) merely using Google (a more mature apparatus) as a means of learning to deliver better results to the user. The only way somebody is going to "build a better Google" is by training their data on what makes Google so popular to begin with. Brave is able to do this is a secure and private manner.


So Google taking data from sites and putting it on theirs is "stealing" but Brave doing that is different and Brave taking results from Google is considered "learning" or "training"? Hmm... I'm beginning to think people just hate Google because it's "cool" to have a negative opinion about it here.


There is a difference:

Google is taking page content from sites and putting it on their own search result page; Brave is taking search-result links from Google and putting them on their own search result page.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: