Deconstructing SPF Records with Rust

 Posted on April 8, 2021  |  12 min  |  2499 words  |  Bas-Man

Hi. If you have visited my blog before you might have seen my series on using the trust-dns-resolver crate. As part of that series I looked at querying DNS TXT records. During the process I was reminded of SPF records. Something I used to deal with in a previous position. And this got me to thinking about using SPF records as a way to learn more about working with rust.

Keep in mind I am new to rust and these will just be some of my thoughts as I work through the task I have given myself. This will not be a production project, merely a learning experience. I will not be documenting all the code here. I will give a reference to the git repository at the end of this article.

TL;DR;

Some caveats

  • I will not be implementing a full solution to deconstruct an SPF record at this time.
  • Not all edge cases may be covered, though I will do my best to do so.
  • I will initially only be looking at: redirect, include, ip4 and ip6 records contained within the SPF record.
  • If there is no qualifier present, it is assumed to be + which equates to a Pass
  • During this phase I will not be looking at using generics as I want to understand the basics of using structs
  • I will not be writing any code that validates that the SPF record is syntactically correct.
  • The code is going to be fairly ugly and will get polished as I go through.
  • I am not looking at SPF2 (pra/mfrom)

Some SPF rules I am fairly familiar with, though I could be mistaken. (smile)

  • There can only be one redirect in a single SPF record.
  • The qualifier is optional
  • There can be multiple instances of ip4, ip6 and include
  • There can only be a single a and mx when they are not followed by a : character.
  • With the exception of a redirect there must an all at the end of an SPF record.

Let’s get started

I am not going to start off with how to create a rust project. There is plenty on that out there. And if you read my previous series on Working with the trust-dns-resolver Crate, you already know how to get started. In fact this is an extension to that series.

Structs

Following the basic rules I listed above each mechanism can have an optional qualifier. This will need to be dealt with. And each mechanism can occur multiple times in most cases.

The Include Struct

An include can be represented in the following ways: (Not an exhaustive list)

  • include:_spf.example.com
  • +include:_spf.otherdomain.com
  • -include:_spf.somedomain.com
#[derive(Default, Debug, Clone)]
struct Include {
    qualifier: char,
    txt: String,
}

Code Explanation

  • I will store the qualifier as a char, this will allow me to hold +,-,~, or ?. Alternatively I could create an enum type. But this is over kill since I am just trying to learning at this point.
  • I will store the _spf.example.com portion in a String which lives on the heap in memory.
  • To save on some code, I am asking the compiler to generate Default, Debug and Clone for me using the #[derive]

Implementing the Include Struct

Rust is interesting in that it allows us to create functions which are tied to a struct. In my mind this is some what like the struct is acting as an object, and the functions are then used as methods of the struct.

impl Include {
    fn new(qualifier: char, txt: String) -> Self {
        Self { qualifier, txt }
    }
    fn as_mechanism(&self) -> String {
        let mut txt = String::new();
        if self.qualifier != '+' {
            txt.push(self.qualifier);
        }
        txt.push_str("include:");
        txt.push_str(self.txt.as_str());
        txt
    }
    fn is_pass(&self) -> bool {
        if self.qualifier == '+' {
            true
        } else {
            false
        }
    }
    fn is_fail(&self) -> bool {
        if self.qualifier == '-' {
            true
        } else {
            false
        }
    }
    fn is_softfail(&self) -> bool {
        if self.qualifier == '~' {
            true
        } else {
            false
        }
    }
    fn is_neutral(&self) -> bool {
        if self.qualifier == '?' {
            true
        } else {
            false
        }
    }
}


<div data-nosnippet>
  <p>The function <code>new</code> allows us to instantiate and struct with the passed values. We can call it in the following ways</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-rust" data-lang="rust"><span class="line"><span class="cl"><span class="kd">let</span><span class="w"> </span><span class="n">var_a</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Include</span>::<span class="n">new</span><span class="p">(</span><span class="sc">&#39;+&#39;</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;_spf.example.com&#34;</span><span class="p">.</span><span class="n">as_string</span><span class="p">());</span><span class="w">
</span></span></span></code></pre></div><ul>
<li>Single quotes can be used here to denote a type of <code>char</code></li>
<li>Text enclosed in double-quotes are technically of type <code>str</code> and since my function definition expects a <code>String</code>, I am transforming the string. This is purely for the example purposes. I could have also easily passed <code>String::from(&quot;_spf.example.com&quot;)</code></li>
</ul>
<p>The <code>is_*</code> functions all simple check the value stored in <code>qualifier</code> and return a boolean.</p>
<p>The <code>as_mechanism()</code> function is able to reconstruct the spf mechanism and return it as a <code>String</code>. If the <code>qualifier</code> is <strong>NOT</strong> a <code>+</code>, it is appended to the string being created. Otherwise it is omitted.</p>

</div>




  
  <ins class="adsbygoogle"
      style="display:block; text-align:center;"
      data-ad-layout="in-article"
      data-ad-format="fluid"
      data-ad-client="ca-pub-9846626266097280"
      data-ad-slot="1585219571">
  </ins>
  <script>
      (adsbygoogle = window.adsbygoogle || []).push({});
  </script>
  



### The Ip4 Struct

An `ip4` can be presented in the following ways: (not an exhaustive list)

- `ip4:157.55.9.128/25`
- `+ip4:ip4:72.14.192.0/18`
- Or as a single host `ip4:X.X.X.X` which would be used as `X.X.X.X/32`

```rust
#[derive(Debug, Clone)]
struct Ip4 {
    qualifier: char,
    ip: IpNetwork,
}
```

Here I am only using `Debug` and `Clone` as I am implementing my own `Default` merely for learning purposes. The astute reader might already conclude that this struct is is very similar to `Include` with the only real difference being `String` and `IpNetwork`. IpNetwork is provided by the [ipnetwork](https://crates.io/crates/ipnetwork) crate. I am using this as it provides lots of the basic functions I will need and also many more that would make it useful for other applications.

```rust
impl Default for Ip4 {
    fn default() -> Self {
        Self {
            qualifier: '+',
            ip: IpNetwork::V4("0.0.0.0/0".parse().unwrap()),
        }
    }
}
```

With his `Default` we can call code such as:

```rust
let ip: Ip4 = ip4::default();
```

### Implementing the Ip4 Struct

```rust
impl Ip4 {
    fn new(qualifier: char, ip: IpNetwork) -> Self {
        Self { qualifier, ip }
    }
    fn as_string(&self) -> String {
        self.ip.to_string()
    }
    fn as_mechanism(&self) -> String {
        let mut ip4_string = String::new();
        if self.qualifier != '+' {
            ip4_string.push(self.qualifier);
        }
        ip4_string.push_str("ip4:");
        ip4_string.push_str(self.ip.to_string().as_str());
        ip4_string
    }
    fn as_ip(&self) -> IpNetwork {
        self.ip
    }
    fn is_pass(&self) -> bool {
        if self.qualifier == '+' {
            true
        } else {
            false
        }
    }
    fn is_fail(&self) -> bool {
        if self.qualifier == '-' {
            true
        } else {
            false
        }
    }
    fn is_softfail(&self) -> bool {
        if self.qualifier == '~' {
            true
        } else {
            false
        }
    }
    fn is_neutral(&self) -> bool {
        if self.qualifier == '?' {
            true
        } else {
            false
        }
    }
}
```

As you can see I have had to duplicate a lot of code.... I think `generic` is going to be in the near future.

### The Ip6 Struct

To keep this a little shorter I will just give the basics since it is identical to Ip4

```rust
#[derive(Debug, Clone)]
struct Ip6 {
    qualifier: char,
    ip: IpNetwork,
}

impl Ip6 {
    fn new(qualifier: char, ip: IpNetwork) -> Self {
        Self { qualifier, ip }
    }
}

impl Default for Ip6 {
    fn default() -> Self {
        Self {
            qualifier: '+',
            ip: IpNetwork::V6("FE80::1".parse().unwrap()),
        }
    }
}
```

This is essentially the same as `Ip4` because `IpNetork` is an `enum` of `V4` and `V6`

### Defining the SPF1 Struct

Many of the `mechanisms` in SPF are optional and nicely, rust actually provides the type `Option`. Also a `mechanism` can appear several times, as a `list` and rust provides `Vec` which is a way to store a list of some type.

```rust
#[derive(Default, Debug)]
struct Spf1 {
    source: String,
    include: Option<Vec<Include>>,
    redirect: Option<String>,
    is_redirected: bool,
    a: Option<Vec<A>>,
    mx: Option<Vec<Mx>>,
    ip4: Option<Vec<Ip4>>,
    ip6: Option<Vec<Ip6>>,
    all_qualifier: char,
}
```

#### Explanation of the SPF1 Struct

- source: The original string received from the TXT DNS lookup. This will be deconstructed to populate the SPF struct.
- include: This is an optional list of includes so I am creating a `Vec` of `Include` and then wrapping that inside and `Option` since it might actually be `None`
- `Ip4` and `Ip6` are handle in the same way as `Include`
- redirect: Only ever occurs once in a record. So I am placing it directly within SPF1, but again it is optional, so I wrap it inside an `option`
- is_redirect: Is closely tied to `direct` and simply stores a `bool` value
- all_qualifier: This is a special case. It is only ever not present when the spf record is a `redirect` so I am only ensure that I have a `qualifer` value.

### Implementing SPF1 Struct

```rust
impl Spf1 {
    fn new(str: &String) -> Self {
        Self {
            source: str.clone(),
            include: None,
            redirect: None,
            is_redirected: false,
            a: None,
            mx: None,
            ip4: None,
            ip6: None,
            all_qualifier: '+',
        }
    }
    fn parse(&mut self) {
        let records = self.source.split_whitespace();
        let mut vec_of_includes: Vec<Include> = Vec::new();
        let mut vec_of_ip4: Vec<Ip4> = Vec::new();
        let mut vec_of_ip6: Vec<Ip6> = Vec::new();
        for record in records { // Main loop
            if record.contains("redirect=") {
                let items = record.rsplit("=");
                for item in items {
                    self.redirect = Some(item.to_string());
                    break;
                }
                self.is_redirected = true;
            } else if record.contains("include:") {
                let qualifier_and_modified_str = return_and_remove_qualifier(record, 'i');
                for item in record.rsplit(":") {
                    vec_of_includes.push(Include::new(
                        qualifier_and_modified_str.0,
                        item.to_string(),
                    ));
                    break; // skip the 'include:'
                }
            } else if record.contains("ip4:") {
                let qualifier_and_modified_str = return_and_remove_qualifier(record, 'i');
                if let Some(raw_ip4) = qualifier_and_modified_str.1.strip_prefix("ip4:") {
                    let network: Ip4 =
                        Ip4::new(qualifier_and_modified_str.0, raw_ip4.parse().unwrap());
                    vec_of_ip4.push(network);
                }
            } else if record.contains("ip6:") {
                let qualifier_and_modified_str = return_and_remove_qualifier(record, 'i');
                if let Some(raw_ip6) = qualifier_and_modified_str.1.strip_prefix("ip6:") {
                    let network: Ip6 =
                        Ip6::new(qualifier_and_modified_str.0, raw_ip6.parse().unwrap());
                    vec_of_ip6.push(network);
                }
            } else if record.ends_with("all") {
                self.all_qualifier = return_and_remove_qualifier(record, 'a').0;
            }
        } // End of Main Loop
        if vec_of_includes.len() > 0 {
            self.include = Some(vec_of_includes);
        };
        if vec_of_ip4.len() > 0 {
            self.ip4 = Some(vec_of_ip4);
        };
        if vec_of_ip6.len() > 0 {
            self.ip6 = Some(vec_of_ip6);
        };
    }
    // Code omitted for brevity
}
```

#### Explanation

```rust
let mut spf = Spf1::new(&some_spf_record.to_string());
```

This function takes in a reference, borrows, a string and then internally uses `clone()` to make a copy of the string which then remains valid for the life time of the struct. Is also initialises the variables within the struct.



  
  <ins class="adsbygoogle"
      style="display:block; text-align:center;"
      data-ad-layout="in-article"
      data-ad-format="fluid"
      data-ad-client="ca-pub-9846626266097280"
      data-ad-slot="1585219571">
  </ins>
  <script>
      (adsbygoogle = window.adsbygoogle || []).push({});
  </script>
  



### Deconstructing time (Finally)

Enter the real work horse in this process. It's currently a little monolithic at the moment. But, it's getting the job done.

```rust
spf.parse();
```

- The first four lines set things up.
  - Split the string in to seperate parts based on white space. Placing them into an `iter()`
  - Create our mutable `Vec` varibles for `Include`, `Ip4` and `Ip6`
- Next we start the main loop which will parse each `mechanism`
  - redirect:
    - If the current record contains `rediect=` we use `rsplit()` which returns a new `iter()` and access the right side of the split first. We wrap that result into a `Some()` because Spf1.redirect expects an `Option`. Next we `break` from the loop since we don't need any other parts of this iter(). We are also not concerned about the `qualifier` in this case as it is usually omitted.
  - include:
    - Similar to the `redirect` we do an rsplit() but on ":"
    - Create a new `Include` struct and push that value on to the `vec_of_includes` that was defined at the start of `parse()`
    - Call `break` to exist inner loop
  - `ip6` has an issue. I am using `strip_prefix()` because I am unable to use `rsplit()` due to `ip6` being comprised of multiple `:` characters. This is an issue because I am using `strip_prefix()` on `"ip6:"` This _will_ fail if there is a `qualifer` present in the string.
    - ie: "+ip6:....". To handle this I have a helper function `return_and_remove_qualifier()` which checks for the presence of a `qualifier`, and returns a `tuple`. The tuple contains the `qualifier` and an updated version of original string with the `qualifier` removed if it was present. This ensure that the `strip_prefix()` function will match correctly.
  - ip4/6:
    - Call `return_and_remove_qualifier(original_string, 'i')`
    - `strip_prefix()` returns `Some()`, so we take a peek inside and conditionally create an `Ip6` struct.
    - Push it on vec_of_ip6, I actually do the same process for `ip4`
  - all;
    - If the the `mechanism` ends with `all` we just store the `qualifier` directly into `self.all_qualifier`
  - When we reach the end of the main loop we then conditionally wrap our various `vec_of.*` structs into their own `Some()` and assign them to their matching property in `Spf1`

  ## Test Domain and Test Output.

  ### Test Domains

  I am using the following records at the time of writing.

  | Domain                  | Spf Record                                                                                                                                                                                                                      |
  | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  | gmail.com               | v=spf1 redirect=\_spf.google.com                                                                                                                                                                                                |
  | \_netblocks.google.com  | v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all |
  | \_netblocks2.google.com | v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all                                                                      |
  | hotmail.com             | v=spf1 ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.outlook.com include:spf-b.outlook.com include:spf-a.hotmail.com include:\_spf-ssg-b.microsoft.com include:\_spf-ssg-c.microsoft.com ~all            |

### Test Output (gmail.com)

```zsh
List of TXT records found for gmail.com.
TXT Record 1:
globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8=
TXT Record 2:
v=spf1 redirect=_spf.google.com

Decontructing SPF Record
SPF1: v=spf1 redirect=_spf.google.com

There are no include elements
There are no ip4 networks
There are no ip4 spf records.

Is a redirect: true

redirect: _spf.google.com
mechanism: redirect=_spf.google.com
```

### Test Output (\_netblocks.google.com)

```zsh
List of TXT records found for _netblocks.google.com.
TXT Record 1:
v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/
18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 i
p4:216.239.32.0/19 ~all

Decontructing SPF Record
SPF1: v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.
192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.
0/19 ip4:216.239.32.0/19 ~all

There are no include elements
List of ip4 networks/hosts:
35.190.247.0/24
64.233.160.0/19
66.102.0.0/20
66.249.80.0/20
72.14.192.0/18
74.125.0.0/16
108.177.8.0/21
173.194.0.0/16
209.85.128.0/17
216.58.192.0/19
216.239.32.0/19

List of ip4 mechanisms:
ip4:35.190.247.0/24
ip4:64.233.160.0/19
ip4:66.102.0.0/20
ip4:66.249.80.0/20
ip4:72.14.192.0/18
ip4:74.125.0.0/16
ip4:108.177.8.0/21
ip4:173.194.0.0/16
ip4:209.85.128.0/17
ip4:216.58.192.0/19
ip4:216.239.32.0/19

Is a redirect: false
```

### Test Output (\_netblocks2.google.com) / Not being displayed at this time.

```zsh
List of TXT records found for _netblocks2.google.com.
TXT Record 1:
v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/3
6 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all

Decontructing SPF Record
SPF1: v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:40
00::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all

There are no include elements
There are no ip4 networks
There are no ip4 spf records.

Is a redirect: false
```

### Test Output (hotmail.com)

```zsh
List of TXT records found for hotmail.com.
TXT Record 1:
google-site-verification=gqFmgDKSUd3XGU_AzWWdojRHtW3_66W_PC3oFvQVZEw
TXT Record 2:
v=spf1 ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.outlook.com include:spf-
b.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsof
t.com ~all

Decontructing SPF Record
SPF1: v=spf1 ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.outlook.com includ
e:spf-b.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.mi
crosoft.com ~all

Include Mechanisms:
include:spf.protection.outlook.com
include:spf-a.outlook.com
include:spf-b.outlook.com
include:spf-a.hotmail.com
include:_spf-ssg-b.microsoft.com
include:_spf-ssg-c.microsoft.com
List of ip4 networks/hosts:
157.55.9.128/25

List of ip4 mechanisms:
ip4:157.55.9.128/25

Is a redirect: false
```

You can get a copy of the complete code at the time of this writing [here](https://github.com/Bas-Man/learning-rust-trust-dns-resolver/tree/SPF-INITIAL)

I think I will next work on using `generics`

Thanks for reading.
rust  dns  spf  spf-records  learning  structs 

See also