Red Hat CDK DNS Issues

While not my usual wheelhouse, I've been struggling with a DNS issue with the Red Hat CDK on my home network. Not that I'm totally ignorant in the ways of configuring and running my own BIND daemons for fun and profit, this issue caught me off guard.

I run an internal BIND server on a Raspberry Pi running an Fedora 25 for ARMv7. While it resolved internal network IP addresses perfectly, I didn't have it configured as a caching DNS server for external addresses. Anything "outside the building" (my house) would go through my ISP's DNS servers because that's the list of DNS servers given to any machine on the network, whether by DHCP or manual configuration.

My CDK hosts' /etc/resolv.conf files looked like this:

# Generated by NetworkManager
search <My local domain name>
nameserver <My local BIND server's IP>
nameserver <My ISP router's IP>


Therein lay the failure.

Here's the behavior I saw. When starting the vagrant image, I'd see the following:

Bringing machine 'default' up with 'libvirt' provider...
==> default: Starting domain.
==> default: Waiting for domain to get an IP address...
==> default: Waiting for SSH to become available...
==> default: Registering box with vagrant-registration...
    default: Would you like to register the system now (default: yes)? [y|n]y
    default: username: XXXXXXXXXXX
    default: password:
==> default: Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.
==> default: Registering to: subscription.rhn.redhat.com:443/subscription


At that point, the CDK was running and was available for SSH. However, all DNS resolution attempts failed with "temporary failure in name resolution," including that required for Red Hat Subscription Manager registration and to pull new Docker images for the CDK.

Vagrant and libvirt use dnsmasq to provide a small DHCP service and DNS proxy to hosted containers. However, dnsmasq by default apparently only considers the first nameserver configured for the host. Since my host's first nameserver only resolved internal hostnames, redhat.com names were not resolvable.

The solution was nice and simple, thankfully. Just add a set of forwarders--in my case, my local ISP router and a couple of well-known DNS server farms.

forwarders {
  192.168.1.1;
  8.8.8.8;
  8.8.4.4;
};


Be careful with the BIND configuration options though. With recent DNS attacks causing all kinds of widespread havoc, there's plenty of opportunity for perpetuating someone else's bad behavior.

Inaugural Post

in•te•gral

(ĭn´tĭ·grəl, ĭn·tĕg´rəl)

• adj.
  
  1. Essential or necessary for completeness; constituent: The kitchen is an integral part of a house.
  2. Possessing everything essential; entire.
• n.
  
  1. A complete unit; a whole.

integral. (n.d.) American Heritage® Dictionary of the English Language, Fifth Edition. (2011). Retrieved October 19 2016 from http://www.thefreedictionary.com/integral

Introductions first. As you can probably tell from my profile, my name is Ryan. I'm currently working as an Architect in the Enterprise Integration Practice in NA Emerging Technologies Consulting at Red Hat. The Enterprise Integration Practice is focused on Red Hat JBoss Fuse and A-MQ, including the upstream Apache Camel, Karaf, ActiveMQ, CXF, and associated projects. A Practice Architect's role is split between client consulting engagements and knowledge sharing both within Red Hat and the worldwide developer community.

Intent next. Many avenues exist for knowledge sharing. Technical blogs are a common choice. My intention is to provide relevant content related to building enterprise-scale system integrations with open-source technologies and platforms, not just those with which Red Hat is involved.

In conclusion, a promise. The "in" alliteration ends now....