Making 'included_files' in Netlify Functions Actually Work

During the epic build/rebuild/re-rebuild of this site, I fell down an interesting rabbit hole trying to solve search. The full story is for another post.

This one covers a very specific technical issue that took a bit of digging to figure out.

Shout out to Corey Quinn for the reminder to write this up and share it so no one else needs to stumble through it

The situation

The search feature on this site runs a function that checks the user’s search query against the content. Very straight forward.

The index of the content is a JSON document that is created as part of the Hugo build. You can wire that up easily enough using this great post by Fredrik Johnson.

By default, Netlify functions try to optimize what’s deployed. Netlify has done a great job hiding a lot of the plumbing that runs these functions.

Under the hood it appears that Netlify is using AWS Lambda. If fact you can contact support and have the functions run in your account to unlock a bunch of other use cases

This optimization strips out a bunch of things that you actually might want to include in your function. Supporting data files are the biggest requirement.

To address this, Netlify developed the included_files feature of their build process.

Sometimes Not Included

This feature allows you to explicitly state which files should be included in your function when it deploys.

In your netlify.toml configuration file. You simply add the specific file or pattern of the files you’d like to include.

The documentation for the feature reads clearly and the configuration is quite logical. Reading it through, you come up with a simple approach; if you need a file that isn’t part of your code, make sure it’s references in included_files.

The problem is that any permutation of the file path I tried didn’t work. The file did not appear to be included in the function build and this prevented the function from working.

Solution

To cut to the chase, here how to get the feature to actually work;

1. Paths in the included_files parameter in the netlify.toml file, start in the directory with your netlify.toml file (most frequently the root of your code repo)
2. Paths in the included_files parameter must start with ./
3. When using the files in your function, start the path with ./

An Example

Using a project with the following data structure:

[ repo root ]
netlify.toml
…hugo & other stuff…
/netlify
  /functions
    /a-nodejs-function
      /data
      sample.json

In the netlify.toml file, you’ll need the following:

[functions]
  node_bundler = "esbuild"
  included_files = ["./netlify/functions/a-nodejs-function/data/sample.json"]

In the function, address the file using the full path:

var pathToSampleFile = './netlify/functions/a-nodejs-function/data/sample.json';

Yes, that runs counter to what you would initially expected. However, Netlify’s fantastic support’s explanation clarified everything.

They build the feature so that when you’re looking at your repo (the driver for everything on the platform) the file paths are clear and easier to work with.

…once you know that. 🤣

References

Being a seasoned technical question answerer, I didn’t just reach out to support. I tried a bunch of different approaches and read through any and all things related to the included_files feature on the platform.

Here are the related forum posts:

• We don’t understand how the included_files [functions] parameter works - #9 by existo - Support - Netlify Support Forums
• Understanding the included_files settings for functions - #3 by jen - Support - Netlify Support Forums
• Do Gatsby functions work with the included_files property? - Support - Netlify Support Forums
• Netlify toml - Configuration property functions.included_files must be an object error - #4 by alexjfno1 - Support - Netlify Support Forums

…and the documentation on the feature:

• Optional configuration for functions | Netlify Docs
• File-based configuration | Netlify Docs
• How to Include Files in Netlify Serverless Functions

Netlify’s Support

I’m at the point in my career where I’m more than happy to pay for tooling that saves me time while helping me achieve my goals.

Netlify has a very generous free tier, but one of the major advantages of the Pro tier is that you get access to email support.

When I reached out about this issue, I laid out the issue very similar to this post and also created a sample, stripped down project to show the issue. This sample ran from a public—since removed—GitHub repo that I spun up the request.

Not only did support respond with the solution, they actually opened a PR against my sample repo with a very nice explanation that they had gotten it working with the PR changes deployed and that I could merge the changes and try it out.

That worked. Of course it worked, they literally sent me a working configuration that tI had the ability to test myself.

Furthermore, when I followed up to clarify the edges of the problem space, support was kind enough to articulate the situation, the reasoning, and to make sure that I was comfortable with my understanding of it all.

That type of support experience is very much appreciated and really cemented that I’d made the right choice in providers for this project.

Well done Netlify