As I promised in the preceding post I was going to provide patches to BCC for sockmap support. Ta-da! Patches are here and I've also added a BCC version of the example code.

First attempt revisited

After tracking down the last place needed to be modified to add sockmap support to BCC I can now revisit the first BCC sockmap attempt. As expected, the code is very similar but with some BCC specifis.

const std::string BPF_PROG = R"(
    #include <uapi/linux/bpf.h>
    #include <uapi/linux/if_ether.h>
    #include <uapi/linux/if_packet.h>
    #include <uapi/linux/ip.h>

    BPF_SOCKMAP(sock_map, 2);
    BPF_HASH(ip_map, u64, int, 64);

    int prog_parser(struct __sk_buff *skb)
    {
	  return skb->len;
    }

    int prog_verdict(struct __sk_buff *skb) {
      __u64 ip = skb->remote_ip4;
      __u32 port = skb->remote_port;
      __u64 key = (ip << 32) | port;

      int *idx = ip_map.lookup(&key);
      if (!idx) {
        return SK_DROP;
      }

      return sock_map.sk_redirect_map(skb, *idx, 0);
    }
)";

eBPF programs in BCC can be defined inline in the source code itself, this is very handy since it allows the program to be customizable at runtime very easily and also keeps the program defined as close to the point of use as possible.

You might have noticed there are two odd statements in the above:

int *idx = ip_map.lookup(&key);
...
return sock_map.sk_redirect_map(skb, *idx, 0);

This is some of the BCC magic. The custom BCC clang frontend will actually rewrite the code above to the appropriate C API, to see how it's done for the sk_redirect_map call take a look at sockmap patch here.

static int add_ip(ebpf::BPFHashTable<uint64_t, int> &ip_map, ebpf::BPFSockmapTable &sock_map, const client &from, int idx, const client &to) {
  uint64_t key = (static_cast<uint64_t>(htonl(from.ip())) << 32) | htonl(from.port());

  ebpf::StatusTuple status = ip_map.update_value(key, idx);

  if (status.code() != 0) {
    fprintf(stderr, "%d: %s\n", status.code(), status.msg().c_str());
    return -1;
  }

  int fd = to.fd();
  status = sock_map.update_value(idx, fd);

  if (status.code() != 0) {
    fprintf(stderr, "%d: %s\n", status.code(), status.msg().c_str());
    return -1;
  }

  return 0;
}

Very similar as the previous libbpf implementation except for the use of the BPF.h C++ API instead to update the maps.

int main(int argc, char *argv[]) {
  ebpf::BPF bpf;
  ebpf::StatusTuple status = bpf.init(BPF_PROG);

  if (status.code()) {
    fprintf(stderr, "%d: %s\n", status.code(), status.msg().c_str());
    return -1;
  }

  ebpf::BPFSockmapTable sock_map = bpf.get_sockmap_table("sock_map");
  ebpf::BPFHashTable<uint64_t, int> ip_map = bpf.get_hash_table<uint64_t, int>("ip_map");

  int parser_fd;
  status = bpf.attach_fd("prog_parser", BPF_PROG_TYPE_SK_SKB, parser_fd, BPF_SK_SKB_STREAM_PARSER, sock_map.get_fd());
  if (status.code() != 0) {
    fprintf(stderr, "%d: %s\n", status.code(), status.msg().c_str());
    return -1;
  }

  int verdict_fd;
  status = bpf.attach_fd("prog_verdict", BPF_PROG_TYPE_SK_SKB, verdict_fd, BPF_SK_SKB_STREAM_VERDICT, sock_map.get_fd());

  if (status.code() != 0) {
    fprintf(stderr, "%d: %s\n", status.code(), status.msg().c_str());
    return -1;
  }
  ... // everything else is the same from this point

And finally how to load the program, instantiate the maps and attach the verdict and parser. Again very similar to the libbpf version.