# searchhut.md -rw-r--r-- 2.4 KiB View raw

#Statement regarding SearchHut

SearchHut is a FOSS search engine I have been hacking on for the past week, which was "announced" this morning.

I'm really unhappy with how the roll-out went. It was much too early. I had been discussing the project and sharing access to the development environment for testing and feedback in the SourceHut off-topic IRC channel, which is a relatively small community, with an explicit request not to share details of the project more widely outside of the channel. Someone in this channel misunderstood my request and posted it to Hacker News, and by the time I woke up and saw it the next morning, it was too late to stop.

The search engine is not presently capable of producing useful results for a large variety of queries. The search index, now being rebuilt for production, is only a quarter of the size of my local development index. I also have not been able to verify my assumptions at scale, and I have no clue if we're a few million pages away from 5-10 second search queries and having to rewrite the indexing backend, or if our weighting heuristics are wrong and we're going to have to rebuild the index after some tweaks. I have not had time to test the production hardware to the extent that I was planning, or to do a renewed cost analysis. The site is full of 404 links and documentation stubs, and the form to request new domains, approve or reject such requests, or browse the list of domains, is not implemented. The search query syntax was planned to be extended later on, and the documentation & draft blog post mention several aspirational features which don't actually work as if they were already there.

My typical workflow involves coming up with and implementing a design, then verifying that design by proving it in the crucible of real-world workloads. Until this process is complete, my software is not done, and I don't feel confident shipping it. Without this, I have no basis to establish confidence in the system, and I don't want users to rely on it. Nevertheless, I felt forced to ship this incomplete and unverified software to end-users on short notice.

Consequently, I've decided to take the system offline for the time being. I'm not sure when it will be back; my motivation for this project has evaporated. It went from a fun experiment to a premature nightmare overnight.

The fault is mine: I should have been clearer about my intentions on IRC. I apologise if you have been adversely affected by this decision.