Query Github GraphQL

Searching for a package on Github can be a daunting task, because the returns can be overwhelming, and it can be extremely time consuming to wade through them one by one.

For me, not a single searching strategy is fully working, due to the overwhelming hits that I’m getting. Until I find a good solution.

Keep reading at “Query Github GraphQL


The “no-sugar” style

I want to leave out any frivolous distraction from my writing. The number one meaning of frivolous from OED is “not having any serious purpose or value“. Those are the things I want to eliminate, so that my writing can be succinct, meaning “briefly and clearly expressed, especially of something written or spoken” from OED.

what’s best phrase to describe it? Continue on to The “no-sugar” style

Dbab From Start To Finish

Dbab From Start To Finish

The following introduction is for dbab at or over version 1.3.1, which is incompatible with previous version (1.2.x) as the configuration files have been renamed.

EDIT: This post really looks terrible under the new “improvement” from wordpress. I guess this will be the last time I post technical blogs here. A nicer looking one is available here. The latest version of this article will be available here.

<a name=”advantages”/>

Dbab Advantages

Before dipping into the following details, here are the advantages of using dbab (Dnsmasq-Based Ad-Blocking).

First of all, why this is the best method for ad blocking? Because all other filter based solutions (privoxy, Adblock-Plus, etc) are CPU intensive because of a large quantity of ad urls and page contents need to be pattern matched, and using regular expressions matching is expensive. Adblock Plus, the easiest choice, is actually the worst choice because it is JavaScript based, and it is the slowest. Furthermore, all these method will more or less alter the rendered web page, to remove the ads. This will be even slower, and might cause side effects as well.

The dbab is however, using an entirely different approach for ad blocking. It’s advantages are:

  • Work at the DNS level. Leave the web pages intact, without any pattern matching, string substitution, and/or html elements replacing.
  • Work for your mobile devices as well. Were you previously in the dilemma of choosing ads free or slow response for your mobile devices (iphone, ipad, etc)? Now you don’t. You don’t need to install any thing to your mobile devices for them to enjoy the ad-free browsing experience. Moreover, their browsing speed will increase dramatically on revisited pages/images.
  • Serve instantly. All ads will be replaced by a 1x1 pixel gif image locally served by the Pixelserv server.
  • Maintenance free. You don’t need to maintain the list of ad sites yourself. The block list can be downloaded from pgl.yoyo.org periodically. If you don’t like some of the entries there, you can define your local tweaking that filters them out.
  • Easily customized. It’s trivial to add your own entries to the ad blocking list if the existing ones are not enough for you.


<a name=”static”/>

Static IP

Now let’s start. First, if you haven’t done switching from dynamic IP to static IP yet, check it out first for how to

  • configure the static IP, and
  • add a second static IP address

and check out why to do that as well if you want.

Here is a recap what I did to configure my machine with the static IP and a second one of

cat << EOF > /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
allow-hotplug eth0

# Use static IP instead of dhcp
iface eth0 inet static
# add a 2nd ip address
post-up ip addr add dev eth0
pre-down ip addr del dev eth0

/etc/init.d/network-manager restart
/etc/init.d/networking restart

% ip addr show dev eth0
inet brd scope global eth0
inet scope global secondary eth0

For details and troubleshooting, refer back to the above
switching from dynamic IP to static IP document.

<a name=”plan”/>

The Plan

Once we have our second IP address, the reset of the steps are:

  1. Install & configure DNSmasq.
  2. Remove all existing ad blocking tools if you have any.
  3. Stop your local web server temporarily if you have any.
  4. Before installation dbab, go and visit some websites which have ads on their pages such as “yahoo”, “abcnews” or anything, then
  5. Install & configure the dbab package.
  6. Restart your local web server if you have any.
  7. Now, visit those pages again in different tabs to see if the ads are removed :-).
  8. Install squid caching server, nothing unusual about that.
  9. Setup auto proxy for everyone and every tool to use the ads-blocking web caching server.

That shall be it. Mission accomplished.

Details to follow. But please be warned, as there are so many pieces tied together, and thus so many things to configure, the following steps are long. So be warned and be prepared.
<a name=”install”/>

Install & Configure DNSmasq and Dbab

To install DNSmasq and Dbab

% apt-get update
% apt-get install dnsmasq
% apt-get install dbab

<a name=”configure-dnsmasq”/>

Configure DNSmasq

To configure DNSmasq:

cp /usr/share/doc/dbab/dbab-dnsmasq.service.conf /etc/dnsmasq.d
cp /usr/share/doc/dbab/dbab-dnsmasq.intranet.conf /etc/dnsmasq.d

The dbab-dnsmasq.service.conf provides basic dnsmasq service configuration. It’s content is pretty standard and consistent across all installations, so you don’t need to make any changes to it. The dbab-dnsmasq.intranet.conf however, reflects how exactly your intranet is configured. What provided is just a boilerplate, of which every content should be customized. I.e., from the below listing, we can see that the ISP DNS server address, the dhcp lease range, the local-net domain name and the dhcp hosts should all be customized. Edit /etc/dnsmasq.d/dbab-dnsmasq.intranet.conf to reflect your true intranet configuration.

# == DNS from ISP

# == Dhcp lease (start,end,leasetime)
# supply the range of addresses available for lease and optionally
# a lease time. If you have more than one network, you will need to
# repeat this for each network on which you want to supply DHCP
# service.

# == Domain for dnsmasq. this is optional, but if it is set, it
# does the following things.
# 1) Allows DHCP hosts to have fully qualified domain names, as long
# as the domain part matches this setting.
# 2) Sets the "domain" DHCP option thereby potentially setting the
# domain of all systems configured by DHCP
# 3) Provides the domain part for "expand-hosts"

# == Dhcp hosts.
# dhcp-host=00:28:58:3A:EB:A1,,computer2,infinite
# ^ ^ ^ ^
# MAC IP Address hostname lease time
# E.g.,

Now test it:

dig @ google.ca

/etc/init.d/dnsmasq restart

dig @ google.ca

echo 'nameserver' > /etc/resolv.conf

dig google.ca

This is heavily-simplified version. For details and troubleshooting refer to:

<a name=”configure-dbab”/>

Faq: dnsmasq: setting capabilities failed

If for any reason that you test dbab under docker and you get the following error when starting dnsmasq (say with service dnsmasq start):

% service dnsmasq start
[….] Starting DNS forwarder and DHCP server: dnsmasq
dnsmasq: setting capabilities failed: Operation not permitted

The fix is to tell dnsmasq to run as root by adding user=root to /etc/dnsmasq.conf:

cp /etc/dnsmasq.conf /tmp
sed -i '/^#user=/s/$/\nuser=root/' /etc/dnsmasq.conf
diff -wU1 /tmp/dnsmasq.conf /etc/dnsmasq.conf

# then
service dnsmasq start

Ref: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=514214

Configure Dbab

The Dbab comes with a simple local configure using the second static IP address. To configure dbab to work with a local web server:

  1. Stop dbab-svr service
  2. Change the IP address that dbab uses to the second IP address
  3. Start dbab-svr service
  4. Start your local web server again if you have any. You may need to limit its listening port from to your first static IP as well if necessary.

In details, do the following as root, again assuming that the server’s own IP address is, and its second IP is The second IP will be used for the dbab service (WPAD & pixelserv).

# (run the following as root)

# stop dbab service
/etc/init.d/dbab stop

# use the second IP for dbab-svr to listens on
ip -f inet addr show eth0 | awk '/inet /{print $2}' | sed 's|/.*$||; 1d' | sudo tee /etc/dbab/dbab.addr
# verify its content before moving on
cat /etc/dbab/dbab.addr
# if it is not what you intent it to be, correct it with your text editor
# or, set it manually (with a different IP address)
echo | sudo tee /etc/dbab/dbab.addr

# update ad blocking list with the second IP address

# OPTIONAL! do the following only if you have squid caching server
# and you want to enable automatic WPAD service
hostname | tee /etc/dbab/dbab.proxy
# NB, if your squid caching server is on a different server, do this instead
echo my_squid_server_name | tee /etc/dbab/dbab.proxy
# then,
# Again verify everything here before moving on because script might not be
# 100% time correct. Manually tweaking is inevitable sometimes.

# restart DNS & DHCP
/etc/init.d/dnsmasq restart

# re-start dbab service
/etc/init.d/dbab start

# re-start your local web server again if you have any

# optional, only when dbab will not auto start on boot up
update-rc.d dbab defaults

That’s it. We’re done.

Faq: How to blacklist those bad sites?

All these started because there were one time that the top of google hits are often crammed with rubbish sites. I.e., those sites that contains nothing but key words merely to be listed on top of google hits. These sites are called content-farming sites, and goolge has been constantly fighting with them (Google’s Farmer Update at the end of February, 2011):

“So-called content farms such as Demand Media and Associated Content, both routinely vilified for churning out shabbily produced, keyword-loaded content that often secured top listings at Google, were penalized severely.” [1]

[1] http://www.websitemagazine.com/content/blogs/posts/pages/crop-devastation-google-s-farmer-update-retools-rankings.aspx

Yet, there are still many content-farming sites that fall through the crack or revamp again. I was so annoyed that, instead of waiting for google to deal with them, I took the matter into my own hand. Here is the updated version that makes use the dbab package to block them:

First, gather a list of those rubish sites:

cat >> /etc/dbab/dbab.list+

The result will look something like this:

$ cat /etc/dbab/dbab.list+

Then, convert the list to be used by DNSmasq:


Those bad sites are now blocked by DNSmasq, after restarting it:

/etc/init.d/dnsmasq restart

That’s it. Next time if you accidentally click into those sites,
You will see a blank page, which loads instantly, with the
following as the page title:

(GIF Image, 1×1 pixels)

Then you know you’ve stumbled into sites that you should have avoided.

Faq: How to whitelist some sites?

First see what exactly was listed in the pgl.yoyo.org list. E.g., to enable http://www.googleadservices.com, merely putting http://www.googleadservices.com into etc/dbab/dbab.list- won’t help, because:

$ grep googleadservices /etc/dnsmasq.d/dbab.*

I.e., we should put in googleadservices.com instead of http://www.googleadservices.com.

Now suppose we need to whitelist googleadservices.com and urlcash.net, here is how to do:

echo ‘googleadservices.com’ > /etc/dbab/dbab.list-
echo ‘urlcash.net’ >> /etc/dbab/dbab.list-

grep googleadservices /etc/dnsmasq.d/dbab.*

service dnsmasq restart

dig http://www.googleadservices.com

It should show real IP instead of

Switching Over to DNSmasq Service

To make the above changed configuration take effect, dnsmasq must be
restarted (because sending SIGHUP to the dnsmasq process will only cause it
to empty its cache and then re-load /etc/hosts and /etc/resolv.conf):

/etc/init.d/dnsmasq restart

But before doing that, we need to disable (DSL) router’s dhcp and dns
services, because (DSL) router would normally act as both dhchp and dns server
for the most cases. if I dedicate a dnsmasq server for both dhcp and dns
servers, I have to disable DHCP on my router so only my own dnsmasq server
responds to DHCP requests. For DNS, the DHCP response can give the IP
address of the DNS for the clients to use.

Having restarted the dnsmasq service, we still can’t test anything about DNS leases because dnsmasq doesn’t return results for dns query until after it has actually served out the address. So we can’t prove anything until after a DHCP/DNS request is made.

Again, for details and troubleshooting refer to:

<a name=”squid”/>

Local Caching Server

Now it is time to make it easy for anyone visiting your home to enjoy you fast local squid caching server. Let’s continue on that trend to auto proxy setting. I.e., DNSMasq gets DHCP and DNS together, and the dbab brings them both and ad blocking together, and now let’s move a step further to bring squid and auto proxy setting into the picture and into the harmony.


To recap, we need a dedicated server in the SOHO environment for

All of them are hosted on a single machine. This is a typical and reasonable configuration, because even with all above, the machine does not need to be a powerful or even a fast one. Mine is a Pentium 5, with 4G of RAM and 300G of disk space, and have a web server, a time server, a printing server, an email server and a SSH server installed as well, along with the DHCP, DNS & web caching server, and it has more than enough power to handle everything.

So install the squid caching server on this dedicated SOHO server as normal, and start it. The dbab should have already properly configured to use it. See above listing for how “to enable automatic WPAD service”.


To check ad blocking, revisit in new tabs those pages you just visited that’s full of ads, and compare the differences, or check out the following urls, which are automatically blocked by the dbab-get-list command:


To check your automatic proxy setting, use:

$ curl http://wpad/wpad.dat
function FindProxyForURL(url, host) { return “PROXY mysohosvr:3128; DIRECT”; }

The http://wpad/wpad.dat will always be the same regardless how your servers are called, but mysohosvr shall be the real name of your squid caching server.

To check your automatic proxy results, first set up your browser to use WPAD, then on your SOHO server do the following before visiting any pages:

tail -f /var/log/squid3/access.log

If the places you are visiting show up in the access log, then everything is working. Now fire up your iphone or ipad to visit some sites. As long as your iphone or ipad is using WIFI from your SOHO network, their visit will be cached as well. Or at least so I read. Check the access log to verify. As for Android, sorry, while iphone or ipad are playing by the rules to set proxy automatically from WPAD, Android isn’t. You have to set its proxy manually. Visit some pages with some very-slow-loading pictures, and they visit them again, the picture loading speed will be dramatically faster, especially if your wireless device is not super fast (like mine).

If AOK, you may want to setup a cron job to update the block list on a weekly/monthly basis. E.g.:

ln -s /usr/sbin/dbab-get-list /etc/cron.monthly/