caching with docker-registry

Docker based systems are great, but if you live in New Zealand (and in rural New Zealand, like I do), repeated docker pulls to the Internet can be a little painful. Fortunately you can configure docker-registry to locally cache.

First install docker-registry and configure it as a proxy.

# apt-get install docker-registry
# vi /etc/docker/registry/config.yml
# cat /etc/docker/registry/config.yml 
version: 0.1
log:
fields:
service: registry
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /var/lib/docker-registry
delete:
enabled: true
http:
addr: :5000
headers:
X-Content-Type-Options: [nosniff]
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3
proxy:
remoteurl: https://registry-1.docker.io
# service docker-registry restart

Now configure docker to use the proxy.

# vi /etc/docker/daemon.json 
# cat /etc/docker/daemon.json 
{
"storage-driver": "overlay2",
"registry-mirrors": ["http://localhost:5000"]
}
# service docker restart

Now when you do a docker pull, the cache will be used if the latest image is available locally.

# docker pull faucet/gauge
Using default tag: latest
latest: Pulling from faucet/gauge
Digest: sha256:26e2c56800df26deccc6a1d8377c697d7ca2a38cf2d8fc52739bb6b634028298
Status: Image is up to date for faucet/gauge:latest
docker.io/faucet/gauge:latest
# grep docker-registry /var/log/syslog|grep gauge
Nov 5 22:29:21 finf-gw docker-registry[29624]: time="2019-11-05T22:29:21Z" level=info msg="Challenge established with upstream : {https registry-1.docker.io /v2/ %!s(bool=false) } &{{{%!s(int32=0) %!s(uint32=0)} %!s(uint32=0) %!s(uint32=0) %!s(int32=0) %!s(int32=0)} map[https://registry-1.docker.io:443/v2/:[{bearer map[realm:https://auth.docker.io/token service:registry.docker.io]}]]}" go.version=go1.8.1 http.request.host="localhost:5000" http.request.id=fb8d44dd-193a-4a18-8625-35dc3ea015eb http.request.method=GET http.request.remoteaddr="[::1]:49450" http.request.uri=/v2/faucet/gauge/manifests/latest http.request.useragent="docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 (linux))" instance.id=9bdb53c3-1d48-4015-9708-8f1ddc816651 vars.name=faucet/gauge vars.reference=latest version=v2.6.2+debian
Nov 5 22:29:23 finf-gw docker-registry[29624]: time="2019-11-05T22:29:23Z" level=info msg="Adding new scheduler entry for faucet/gauge@sha256:26e2c56800df26deccc6a1d8377c697d7ca2a38cf2d8fc52739bb6b634028298 with ttl=167h59m59.999997987s" go.version=go1.8.1 instance.id=9bdb53c3-1d48-4015-9708-8f1ddc816651 version=v2.6.2+debian
Nov 5 22:29:23 finf-gw docker-registry[29624]: time="2019-11-05T22:29:23Z" level=info msg="response completed" go.version=go1.8.1 http.request.host="localhost:5000" http.request.id=fb8d44dd-193a-4a18-8625-35dc3ea015eb http.request.method=GET http.request.remoteaddr="[::1]:49450" http.request.uri=/v2/faucet/gauge/manifests/latest http.request.useragent="docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 (linux))" http.response.contenttype=application/vnd.docker.distribution.manifest.v2+json http.response.duration=3.545613795s http.response.status=200 http.response.written=1582 instance.id=9bdb53c3-1d48-4015-9708-8f1ddc816651 version=v2.6.2+debian
Nov 5 22:29:23 finf-gw docker-registry[29624]: ::1 - - [05/Nov/2019:22:29:20 +0000] "GET /v2/faucet/gauge/manifests/latest HTTP/1.1" 200 1582 "" "docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 \(linux\))"
Nov 5 22:31:51 finf-gw docker-registry[29624]: time="2019-11-05T22:31:51Z" level=info msg="response completed" go.version=go1.8.1 http.request.host="localhost:5000" http.request.id=013a90e2-11e3-44d8-8875-03aca0c54be5 http.request.method=GET http.request.remoteaddr="[::1]:49702" http.request.uri=/v2/faucet/gauge/manifests/latest http.request.useragent="docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 (linux))" http.response.contenttype=application/vnd.docker.distribution.manifest.v2+json http.response.duration=2.237893853s http.response.status=200 http.response.written=1582 instance.id=9bdb53c3-1d48-4015-9708-8f1ddc816651 version=v2.6.2+debian
Nov 5 22:31:51 finf-gw docker-registry[29624]: ::1 - - [05/Nov/2019:22:31:49 +0000] "GET /v2/faucet/gauge/manifests/latest HTTP/1.1" 200 1582 "" "docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 \(linux\))"

FAUCET at Sandia National Laboratories: Multivendor FAUCET Implementation for Cloud Development.

Tim Toole – Sandia National Laboratories
Given prior successes at SC18, a group at Sandia transitioned a small lab from using traditional switching and routing to near-100% OpenFlow 1.3 architecture using FAUCET. This talk discusses our use case, goals, and our journey from operating a traditional network to managing a multivendor SDN environment with FAUCET. SNL is managed and operated by NTESS under DOE NNSA contract DE-NA0003525.

Slides and video.

Injecting services into the dataplane with coprocessing

With the coprocessor feature, FAUCET allows an external NFV host to receive packets from the dataplane, inject arbitrary packets, or both. In particular, coprocessing allows you to inject services into the dataplane, or even override services on hosts already present.

In this example, we have a real host 192.168.2.1 (a router) which has an ssh service. We have a host on the network, 192.168.2.5. 192.168.2.5 may do anything it likes, but if it should try to connect to ssh on 192.168.2.1, we want to redirect that connection to a fake service running on the coprocessor server.

First we will need to tell FAUCET to intercept the service using an ACL, which will be applied to the test host’s port. Port 18 is the coprocessor port, and port 13 is the test host.

acls:
  coprocessssh:
    - rule:
        dl_type: 0x806
        actions:
          allow: 1
          output:
            ports: [18]
    - rule:
        dl_type: 0x800
        ip_proto: 6
        tcp_dst: 22
        actions:
          output:
            ports: [18]
    - rule:
        actions:
          allow: 1
vlans:
  trusted:
    vid: 2
dps:
  dp:
    interfaces:
      13:
        native_vlan: trusted
        acls_in: [coprocessssh]
      18:
        coprocessor: {strategy: vlan_vid}

FAUCET will now redirect port 22 to the coprocessor port. We will now need to set up an OVS switch on the coprocessor so we can start fake services, and allow ARP to work in parallel with the real host. The fake host will need to use the same MAC address as the real one, which won’t cause a conflict because FAUCET is intercepting just the TCP service and FAUCET knows not to re-learn the real host on the coprocessor port.

Note the OVS actions for setting VID to 4098 – that’s the VID for the trusted VLAN (2) or’d with the VID present bit (4096). That will cause packets from the coprocessor to have the right VID to be switched back into the trusted VLAN.

root@coprocessor:/home/pi# cat setupcopro.sh 
#!/bin/sh

NFVINT=enx0023565c8859
FAKEHW=aa:56:63:87:9e:e9
FAKEIP=192.168.2.1/24
NS=copro

ip netns add $NS
ifconfig $NFVINT up
ip link add dev fake0 type veth peer name fakeovs0
ifconfig fakeovs0 up
ip link set fake0 netns $NS
ip netns exec $NS ifconfig fake0 hw ether $FAKEHW $FAKEIP up
for i in $NFVINT fakeovs0 ovs-system ; do
  echo 1 > /proc/sys/net/ipv6/conf/$i/disable_ipv6
done
ifconfig ovs-system 0.0.0.0

ovs-vsctl del-br br0
ovs-vsctl add-br br0
ovs-ofctl del-flows br0

for i in $NFVINT fakeovs0 ; do
  ovs-vsctl add-port br0 $i
done

ovs-ofctl add-flow -OOpenFlow13 br0 "in_port=1,actions=output:2"
ovs-ofctl add-flow -OOpenFlow13 br0 "in_port=2,vlan_tci=0x0000/0x1fff,ip,tcp,actions=push_vlan:0x8100,set_field:4098->vlan_vid,output:1"
ovs-ofctl add-flow -OOpenFlow13 br0 "in_port=2,vlan_tci=0x0000/0x1fff,arp,actions=push_vlan:0x8100,set_field:4098->vlan_vid,output:1"

Now we will start a fake service on the coprocessor and connect to it from the test host.

root@coprocessor:/home/pi# cat fakessh.sh 
#!/bin/sh

while /bin/true ; do echo completely awesome ssh service| ip netns exec copro nc -l 192.168.2.1 22 ; done
root@coprocessor:/home/pi# ./fakessh.sh
pi@pi8021x:~ $ telnet 192.168.2.1 22
Trying 192.168.2.1...
Connected to 192.168.2.1.
Escape character is '^]'.
completely awesome ssh service
^]
telnet> close
Connection closed.

That’s it. We could run other services on the fake host – for example to broadcast CDP to impersonate a router, for example, or redirect other services.

FAUCET dev workshop at Cyber Toa Wellington

I will be hosting a developer workshop (including how to get started with FAUCET) at Cyber Toa, 4pm, October 7th in Wellington. I’ll cover how to get up and running, an architectural overview of the FAUCET controller, and how to do development.

If you have any questions or topics you’d like me to include, please do send me email!

Please do register (depending on numbers we may use a larger venue).

FAUCET PlugFest 2019 topology

This year we’ll be combining high availability (in the form of multiple controllers, and switches with multiple redundant links), 802.1X authentication, and distributed monitoring. You can find out more and follow developers on the PlugFest announcement list.

We’ll also be using a C64 with Ethernet interface as a test client – because as everyone knows, a network isn’t enterprise grade until you can get online with GeoLink.