WURFL Integration

Hydrolix provides automatic WURFL (Wireless Universal Resource FiLe) integration. Supply a ScientiaMobile license key and a User-Agent string in your data to automatically decorate table rows during ingest with over 600 WURFL device capabilities.

For more information about WURFL and ScientiaMobile, read our blog article.

WURFL integration is provided via Hydrolix's Auto Values system and a few extra settings in the transform.

Specify Settings⚓︎

Add a "wurfl" entry to the settings section of each transform where you’ll be ingesting WURFL data. These are required fields:

Name	Description
`enabled`	Enables the ScientiaMobile WURFL integration.
`update_url`	Specifies your ScientiaMobile-provided update URL. This included device lookup data and your license.
`header_mapping`	Tells Hydrolix which key in your data specifies the `User-Agent` to facilitate device lookup.

For example, in your transform:

WURFL settings configuration

    "settings": {
        // <your other settings here>
        "wurfl": {
            "enabled": true,
            "update_url": "https://data.scientiamobile.com/{YOUR-SCIENTIAMOBILE-LICENSE-KEY-HERE}/wurfl.zip",
            "header_mapping": {
                "User-Agent": "user_agent"
            }
        }
    }

This example

Enables WURFL.
Configures the update_url to point to your licensed WURFL definition.
Sets the header_mapping key to be a map of HTTP headers that WURFL will use for classification.

Include Fields⚓︎

Provide a mapping in your transform schema for each field you want to include in your Hydrolix table. Here's an example for the field "physical_screen_width".

WURFL field mapping example

{
  "name": "physical_screen_width",
  "datatype": {
    "type": "string",
    "index": true,
    "source": {
      "from_automatic_value": "physical_screen_width"
    }
  }
}

Missing fields⚓︎

If your results are partially decorated, check your ScientiaMobile license level. Some fields are only available with Premium plans from ScientiaMobile.

Initially, undecorated rows are recorded until the ScientiaMobile libraries has completely loaded. This preserves Hydrolix's throughput even if the WURFL capabilities are not yet ready to use.

Example⚓︎

If you have simple CSV-formatted data which includes a timestamp and the contents of a User-Agent header:

Sample CSV data with User-Agent strings

1698438520,"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36"
1698438522,"Mozilla/5.0 (Linux; Android 13; SM-S901U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Mobile Safari/537.36"
1698438524,"Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Mobile Safari/537.36,gzip(gfe)"
1698438527,"Mozilla/5.0 (Linux; Android 12; Redmi Note 9 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Mobile Safari/537.36"
1698438529,"Mozilla/5.0 (Linux; Android 12; DE2118) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Mobile Safari/537.36"
1698438530,"Mozilla/5.0 (iPhone12,1; U; CPU iPhone OS 13_0 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Mobile/15E148 Safari/602.1"
1698438534,"Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1"
1698438536,"Mozilla/5.0 (Apple-iPhone7C2/1202.466; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543 Safari/419.3"
1698438537,"Mozilla/5.0 (Windows Phone 10.0; Android 6.0.1; Microsoft; RM-1152) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Mobile Safari/537.36 Edge/15.15254"
1698438538,"Mozilla/5.0 (Linux; Android 11; Lenovo YT-J706X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36"
1698438541,"Mozilla/5.0 (Linux; Android 6.0.1; SGP771 Build/32.2.A.0.253; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/52.0.2743.98 Safari/537.36"
1698438543,"Roku4640X/DVP-7.70 (297.70E04154A)"
1698438546,"Mozilla/5.0 (PlayStation; PlayStation 5/2.26) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0 Safari/605.1.15"

This transform uses WURFL and the user_agent field to automatically decorate new fields in your tables with device information from ScientiaMobile's list of WURFL device capabilities:

Complete WURFL transform example

{
  "name": "wurfl-test-transform-csv",
  "type": "csv",
  "settings": {
"wurfl": {
  "enabled": true,
  "update_url": "https://data.scientiamobile.com/{YOUR-SCIENTIAMOBILE-LICENSE-KEY-HERE}/wurfl.zip",
  "header_mapping": {
    "User-Agent": "user_agent"
  }
},
"output_columns": [
  {
    "name": "ts",
    "datatype": {
      "type": "epoch",
      "index": false,
      "primary": true,
      "source": {
        "from_input_index": 0
      },
      "format": "seconds",
      "resolution": "second"
    }
  },
  {
    "name": "user_agent",
    "datatype": {
      "type": "string",
      "source": {
        "from_input_index": 1
      }
    }
  },
  {
    "name": "form_factor",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "form_factor"
      }
    }
  },
  {
    "name": "advertised_device_os",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "advertised_device_os"
      }
    }
  },
  {
    "name": "advertised_device_os_version",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "advertised_device_os_version"
      }
    }
  },
  {
    "name": "av1_hardware_decode_level",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "av1_hardware_decode_level"
      }
    }
  },
  {
    "name": "resolution_width",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "resolution_width"
      }
    }
  },
  {
    "name": "resolution_height",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "resolution_height"
      }
    }
  },
  {
    "name": "chipset",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "chipset"
      }
    }
  },
  {
    "name": "ram",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "ram"
      }
    }
  },
  {
    "name": "gpu_model",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "gpu_model"
      }
    }
  },
  {
    "name": "is_connected_tv",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "is_connected_tv"
      }
    }
  },
  {
    "name": "is_ott",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "is_ott"
      }
    }
  },
  {
    "name": "release_msrp",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "release_msrp"
      }
    }
  },
  {
    "name": "release_date",
    "datatype": {
      "type": "string",
      "source": {
        "from_automatic_value": "release_date"
      }
    }
  }
]
  }
}

The resulting rows in your Hydrolix table will contain information like this:

Example WURFL enriched output

{
"advertised_device_os": "Android",
"advertised_device_os_version": "11",
"av1_hardware_decode_level": "-1",
"chipset": "Helio G90T",
"form_factor": "Tablet",
"gpu_model": "ARM Mali G76 3EEMC4",
"is_connected_tv": "false",
"is_ott": "false",
"ram": "8GB",
"release_date": "2021_june",
"release_msrp": "548",
"resolution_height": "1200",
"resolution_width": "2000",
"ts": "2023-10-27 20:38:58",
"user_agent": "Mozilla\/5.0 (Linux; Android 11; Lenovo YT-J706X) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/96.0.4664.45 Safari\/537.36"
},