~?~

~?~

~?~

~?~

Try it out Features About Jobs Blog Try it out Sign In Periscope plugs directly into your databases and lets you run, save and share analyses over billions of data rows in seconds. Sign up Speed up your analyses by up to 10,000X with Periscope's statistical sampling, pre-emptive in-memory caching, and other techniques. Periscope keeps all your charts and dashboards up to date all the time. Never run the same query twice. Periscope plugs directly into your database. No ETL, no uploading, no copying, no hassle. Learn more Matthew Perkins Product Manager, The Climate Corporation “At Climate we use Periscope to help us quickly find out what farmers are doing with our product. Our decisions are quick and data-driven because Periscope gives us the ability to instantly generate insights from SQL queries.” Emily Greer ...

Features About Jobs Blog Sign Up Sign In The High-Performance SQL Blog Subscribe Thank you HyperLogLog in Pure SQL 29 May 2014 Earlier we showed how to make count distinct 50x faster with subqueries . Using probabilistic counting we'll make count distinct even faster, trading a little accuracy for the increase in speed. We'll optimize a very simple query, which calculates the daily distinct sessions for 5,000,000 gameplays (~150,000/day): select date ( created_at ), count ( distinct session_id ) from gameplays The original query takes 162.2s. The HyperLogLog version is 5.1x faster (31.5s) with a 3.7% error, and uses a small fraction of the RAM. Databases often implement count(distinct) in two ways: When there are few distinct elements, the database makes a hashset in RAM and then counts the keys. When there there are too many elements to fit in R...