The requirements for this project were simple, or so I thought: Save recipes from the web in a readily accessible and usable format. I didn't realize until later that I had many more requirements I had not discovered. In the end, valuable lessons were learning about knowing your requirements and... just knowing more?

Timeline​

What have other people done?​

I saw many recommendations for Copy Me That, Paprika App, and plain-old recipe cards, but nothing that satisfied. Many of my requirements were formed at this phase, though they lacked firm statements.

Start somewhere​

Since I'm already immersed in the Google ecosystem, and they've taken my data already, Drive was a familiar choice. At the risk of being shoehorned into a less-than-ideal platform, I created a folder for storing recipes as Google Docs and a Google Sheet for acting as the table of contents. At this point, I considered manually copying and formatting the recipes as a cop-out answer, but I forged ahead into uncharted territory.

Recipes/ 
├── Table-Of-Contents
└── RecipeDocs/ 
	├── Recipe1 
	└── Recipe2

Throw AI at it​

I knew that any AI chat bot would be able to browse to a recipe article and extract the ingredients list and instructions. At the very least, an LLM would be effective at extracting content from raw HTML. I installed Ollama for the first time and played around with gemma3:1b and deepseekr1:1.5b models. While the novelty of running AI on my own computer was exciting, I wasn't any closer to a final product. With this newfound knowledge, and the smallest bit of experience, I was able to form more requirements.

If not on-prem, then cloud​

Cloudflare was the first stop, being the cloud provider that I had the most familiarity with. Even though Cloudflare's compute offerings pale in comparison to the other titans in the industry, finding a solution would mean one less account I had to sign up for.

I came across several offerings that would help me in my search, the first being Email Workers. A serverless function triggered on the receipt of an email. The flow of the solution was beginning to form in my head:

1. Send email with recipe page URL to destination address
2. Parse email to obtain URL
3. Fetch HTML content
4. Prompt LLM to extract recipe info
5. Output to Google Docs

The second offering I discovered was Workers AI. A serverless integration for LLMs with generous free tiers on a known platform. I created a prompt and received a response from the LLM - nice and easy.

The third offering was Browser Rendering, to run headless Chrome for automation and web scraping. Looking back, I should have stopped at the /browser-rendering/content API and sent the raw HTML into an LLM prompt but I was more concerned with finding the "best" solution. I spent the majority of my time working with an integration called Stagehand, an AI powered browser automation library. After code incompatibilities, version conflicts, and overall sub-par performance, the email --> LLM flow achieved a usable output.

Time to integrate with Google​

To fulfil the end goal, the output should be saved to a Google Doc. At some point while looking at the Node.js Drive SDK I asked myself "If I have to integrate with Google anyway, why not just use Google for everything?" This was the beginning of the end, as I bounced around between Google Cloud, Google Workspace and eventually found something I had never heard of before, Google Apps Scripts. There were some limitations, but a new code flow formed in my head:

1. Send email with recipe page URL to personal email
2. Flag email with "Automation" label
3. Parse email to obtain URL
4. Fetch HTML content
5. Prompt LLM to extract recipe info
6. Output to Google Docs

I left all the progress made with the previous solution, somewhat disappointingly, with the hope I could make something that better suits my needs. Within a few hours I had a fully working product. Project available on my GitHub: https://github.com/lostinstarvation/google-recipe-parser

There are downsides to the current approach:

1. No event based trigger, so the execution of the script has to be scheduled
2. Dependent on email rules to correctly identify emails
3. Runs under user account context

Lessons learned​

My requirements looked like this - a stark contrast from the simple statement in the beginning:

1. Have full control over solution to add or remove features
2. Minimize third party integrations and dependencies
3. Free, without paying in privacy
4. Recipes should be exportable or saved in a common format
5. Accessible from desktop or mobile while away or at home
6. Easy to use for multiple people

So, to answer the question, do I need to be better at designing solutions or know how to design solutions better?

• Better at designing solutions: Knowing the right tools for the job comes with time, as you can't know everything from the start
• Design solutions better: One improvement, out of many I'm sure, is to know your requirements

After a long 3 weeks of fumbling around with Microsoft Connected Cache (MCC) and the underlying technologies, I determined that the issue with my deployment ultimately stemmed from the lack of proxy support in the installation scripts. While the root cause of your problem will most likely be different from mine, the steps listed below are a good start to diagnose several possible issues and to aid general understanding the of the application.

I'll be covering the installation of MCC on a Windows 2022 Server with unauthenticated proxy support.

Start your troubleshooting journey here: https://learn.microsoft.com/en-us/windows/deployment/do/mcc-ent-troubleshooting#troubleshooting-cache-node-deployment-to-windows-host-machine

The installation failed on the 'Waiting for MCC container to become available' step:

Step 1: Why is it failing​

"Waiting for MCC container to be downloaded..." is generated by installmcconwsl.ps1 and the following section:

while (!(Wait-ForMccContentToReturn))
{...}

The function 'Wait-ForMccContentToReturn' is located in utilities.ps1:

function Wait-ForMccContentToReturn()
{
    try 
    {
        $Bytes = [system.Text.Encoding]::UTF8.GetBytes("key=value")
        $Web = [net.WebRequest]::Create("http://127.0.0.1:80/mscomtest/cedtest/r20.gif") -as [net.HttpWebRequest]
        $Web.ContentType = "application/x-www-form-urlencoded"
        $Web.Host = "b1.download.windowsupdate.com"
        $Web.Method = "GET"			
        $response = $Web.GetResponse()		
    
        if (($response.StatusDescription).ToUpper() -eq "OK")
        {
            return $true
        }
        else 
        {
            return $false
        }

    }
    catch 
    {
        return $false
    }
}

MCC is a web server running in a container in WSL. The installer is checking to see if a successful web request can be made to localhost over port 80. This web request does not seem to be working.

Step 2: Check networking​

The installation scripts create a firewall and port forwarding rule to take the request from the host VM and route it to the WSL container. More information: https://jwstanly.com/blog/article/Port+Forwarding+WSL+2+to+Your+LAN/.

Confirm the configurations have applied successfully:

Get-NetFirewallRule -DisplayName "WSL2 Port Bridge" | Select DisplayName,Enabled,Direction,PrimaryStatus
netsh interface portproxy show v4tov4

Looks good. Lets see if the cache server is running and able to accept HTTP requests.

Step 3: Check container​

Installing MCC uses a service account to run the WSL instance. Due to security limitations in my environment, I'm not able to run this account interactively. Running WSL as my own account returns no results as the instance is running under a different user. Psexec might be able to work around this limitation, but I set up a task in Task Scheduler with the same service account to run a PowerShell script on demand.

Task scheduler config:

• Action: Start a program
• Program/script: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
• Add arguments: -ExecutionPolicy Bypass -command C:\temp\mccTroubleshooter.ps1 2>&1 > C:\temp\MCCLog.log"

So, by adding some commands to the PowerShell script C:\temp\mccTroubleshooter.ps1, we can see the output in C:\temp\MCCLog.log. For example, the following command will instruct the 'Ubuntu-24.04-Mcc-Base' image running in WSL to run the bash command ip addr in the image.

& 'C:\Program Files\wsl\wsl.exe' -d 'Ubuntu-24.04-Mcc-Base' bash -c "ip addr"

MCC uses Azure IoT containers to handle the requests and management of the application. To check the containers, run the command sudo docker ps. Expect to see three images running.

In my case, only the 'azureiotedge-hub' and 'azureiotedge-agent' images were running. The image named 'msconnectedcacheprod1.azurecr.io' was supposed to be downloaded from the internet by the installer but seemed to have failed.

Step 4: Check networking (again)​

Microsoft provides a list of network endpoints that must be reachable for MCC to function properly https://learn.microsoft.com/en-us/windows/deployment/do/delivery-optimization-endpoints. Pay special attention to the endpoints for 'IoT Edge / IoT Hub communication'. Check your organizations firewall to determine if the listed endpoints are allowed, in addition to performing DNS and ping checks from within the WSL instance.

Example:

 ping global.azure-devices-provisioning.net

You should check all those endpoints manually, but I have a shortcut :).

As mentioned before, MCC is using Azure IoT images. This means that all Azure IoT troubleshooting steps also apply to MCC. We will use the command iotedge check to run a collection of configuration and connectivity checks automatically. More information here: https://learn.microsoft.com/en-us/azure/iot-edge/troubleshoot.

I found, through agonizing trial and error, that the URL representing the on-prem Azure IoT image must be added to the command to correctly check connectivity. This URL is referred to as the 'IoT Hub Hostname'. We can grab that by running the following command in WSL:

sudo docker inspect edgeAgent | grep 'IOTEDGE_IOTHUBHOSTNAME'

Returns a value like this: iotc-00000000-0000-0000-0000-000000000000.azure-devices.net. Use this value in the following command:

sudo iotedge check --verbose --iothub-hostname iotc-00000000-0000-0000-0000-000000000000.azure-devices.net 

If you are using a proxy, you must add a proxy parameter as shown below. Use the URL format http://proxyurl.com:port. The proxy used here is unauthenticated, so additional credentials were not required.

sudo iotedge check --proxy-uri http://proxy.company.com:80 --verbose --iothub-hostname iotc-00000000-0000-0000-0000-000000000000.azure-devices.net

The output of the command will contain a 'Connectivity Check (aziot-identity-service)' section.

host can connect to and perform TLS handshake with iothub AMQP port - Error
    Failed to do TLS Handshake, Connection Attempt Timed out in 70 Seconds
        caused by: Failed to do TLS Handshake, Connection Attempt Timed out in 70 Seconds
        caused by: deadline has elapsed
host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK
host can connect to and perform TLS handshake with iothub MQTT port - Error
    Failed to do TLS Handshake, Connection Attempt Timed out in 70 Seconds
        caused by: Failed to do TLS Handshake, Connection Attempt Timed out in 70 Seconds
        caused by: deadline has elapsed
host can connect to and perform TLS handshake with DPS endpoint - OK

This tells us four things:

• Connection failed over AMQP
• Connection succeeded with HTTPS / Websockets
• Connection failed over MQTT
• Connection succeeded with DPS (Device Provisioning Service)

Azure IoT supports two protocols for communicating back to the Azure IoT hub: MQTT and AMQP. Another option is supported, which is to tunnel AMQP traffic over port 443, called AMQP over WebSockets (AmqpWs). More information: https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-protocols.

MQTT and AMQP ports are being tested as part of the default connectivity checks, but this does not mean they are necessary for MCC to function. In my environment, AMQP and MQTT were failing with TLS timeouts, and AmqpWs appeared to be working when running iotedge check with proxy support. So why was I still getting errors in the application logs? Let's take a look at the AmqpWs setting in the config files.

Step 5: Azure IoT Settings​

Azure IoT settings are found in /etc/aziot/config.toml. I viewed the contents with cat /etc/aziot/config.toml.

[agent.env]
"UpstreamProtocol" = "AmqpWs"
"https_proxy" = ""

The agent.env section shows that no proxy was configured, despite configuring the only available proxy settings (from the web GUI).

This can be confirmed more simply in the proxy settings section of the iotedge check output. I saw this early in the troubleshooting process and did not understand the implications at the time :(

Changing the config file to add the proxy URL fixed all communication issues and allowed MCC to function as expected.

[agent.env]
"UpstreamProtocol" = "AmqpWs"
"https_proxy" = "http://proxy.company.com:80"

This issue was fixed by Microsoft with updates to the install scripts, but the time spent troubleshooting was both a loss of valuable time and a benefit of experienced gained. Just preview feature things!

But the fun continues...

Step 6: Reinstall and... it stopped working?​

Downloading the newest version of the scripts caused the installation to fail once again, this time with a 'certificate verification' error. The updates to the install scripts contained the fix for the proxy issue, but also added new processes.

When the main PowerShell installation script runs, it downloads the latest Linux scripts to run inside WSL. A new script block was added to provisionmcc.sh that validates the certificate of the connection being used. Once again, the unauthenticated proxy in my environment collided with the lack of proxy support in the install scripts, causing terminating errors. The quickest fix I found was to edit out the new section and replace with the below script block after provisionmcc.sh automatically downloads and before provisionmcc.sh is called by the installing script (you have 2-3 minutes to do this).

Remove line 286 - 314:

And replace with:

printDetailMagenta "Calling to retrieve registration endpoint"
GeoResponse=$(curl --insecure -i 'https://geomcc.prod.do.dsp.mp.microsoft.com/geo')
KvEndpointUrl=$( echo $GeoResponse | grep -Po '"KeyValue_EndpointFullUri":.*?[^\\]",' | tr "," "\n"  | cut -d "\"" -f4 | cut -d "/" -f3)
printDetailGreen "Successfully called to retrieve registration endpoint"
printDetailMagenta "Attemtpting to retrieve overrides for this cache node"
DoincCustomerId="doincCustomerId=${CommandLineCustomerId}"
DoincCustomerKey="&doincCustomerKey=${CommandLinePrimaryKey}"
KvVersionUri="${Https}${KvEndpointUrl}${OverrideToolName}${DoincCustomerId}${DoincCustomerKey}"
KvOverrideResponse=$(curl -i ${KvVersionUri})
printDetailGreen "Successfully retrieved overrides for this cache node from $Https$KvEndpointUrl"

If done correctly, the new script block will bypass the certificate check and allow the installation to proceed as normal. Hopefully this issue will also be fixed in upcoming script updates.

MCC for Enterprise and Education is currently in preview, which means features are still under development and may not work as expected (as I’ve discovered). Despite this, the final product has proven to be reliable and effective at caching Windows, Office, and other Microsoft update content locally with minimal management overhead. While the MCC server is only one step in the process to achieve these benefits, it is the most critical if the network health is a priority.

Windows Setup​

I’ll be walking through the Windows setup of MCC in this guide as that fits my use case. Due to the layers of virtualization in Windows Subsystem for Linux (WSL) required to run MCC on Windows, I would recommend exploring the Linux setup of MCC to reduce complexity.

Requirements​

• Licensing and technical prerequisites:
  • https://learn.microsoft.com/en-us/windows/deployment/do/mcc-ent-prerequisites
• Required network endpoints:
  • https://learn.microsoft.com/en-us/windows/deployment/do/delivery-optimization-endpoints

What You'll Need​

• An Azure subscription with a resource group
• A supported system:
  • Windows Server or Windows 11
  • Linux system (Red Hat 8/9 or Ubuntu 24.04)

Installation​

Prepare the Server​

1. I used a Windows Server 2022 VM. As listed in the requirements, nested virtualization must be supported. Confirm this in Task Manager:
2. Install WSL with elevated PowerShell:

wsl --install --no-distribution
wsl --update

3. Reboot the system to ensure WSL is properly registered.

Prepare Azure Resources​

1. In the Azure subscription, register the resource provider "Microsoft.ConnectedCache"
   • Subscription -> Settings -> Resource Providers
2. From your resource group, create a Microsoft Connected Cache for Enterprise resource
3. Create a cache node. This node represents a single server with unique settings and install scripts. You have the option to choose Windows or Linux
4. After creation, open the node to configure the following options:
   1. Cache size (e.g., 100 GB)
   2. Proxy (if required)
      1. Hostname (FQDN only, e.g., proxy.company.com)
      2. Port
5. Save the configuration. The node will auto-generate the provisioning script. This may take a minute and requires a refresh of the page. Review the following instructions:

You have the option of choosing a GMSA, local or domain account. The account you choose will register tasks in Task Scheduler and run the WSL instance as that user.

Run Provisioning Script​

1. Download the provisioning package from the cache node configuration window and extract to a local folder. This package is not unique and can be reused for each cache node.
   • Windows: https://aka.ms/MCC-Ent-InstallScript-WSL
   • Linux: https://aka.ms/MCC-Ent-InstallScript-Linux
2. I used a domain service account. Open an elevated PowerShell window, navigate to the directory where the install scripts were extracted and create the following variables:
   1. $User = "Domain\User"
   2. $myLocalAccountCredential = Get-Credential "Domain\User"
3. Run the provisioning script from the cache node configuration window. This command is unique to each MCC node and cannot be reused to install another node
4. The installer will log to the terminal and the InstallLogs directory under the installation folder specified in the install command. Default is C:\mccwsl01

This is the point where things went wrong for me. Without a good grasp on what was going on, I spent weeks going down rabbit holes and coming up with the wrong solutions. Troubleshooting article coming soon!

Behind the Scenes​

The script tries to:

1. Configure the host with WSL, firewall rules and port forwarding to redirect port 80 requests from the host to WSL
2. Import an Ubuntu image into WSL
3. Download and execute scripts to configure Azure IoT Edge containers in Ubuntu
4. Download the latest MCC container image from Microsoft's repository and install in Ubuntu
5. Verify the node is downloading, caching and serving content successfully

Check the Status​

If the installation is successful, the C:\mccwsl01\InstallLogs\WSL_Mcc_Install_FromRegisteredTask_Status log will show successful image downloads and HTTP 200 content requests.

Review Microsoft's article to verify the cache node is working correctly. Start by running wget from the VM:

wget http://localhost/filestreamingservice/files/7bc846e0-af9c-49be-a03d-bb04428c9bb5/Microsoft.png?cacheHostOrigin=dl.delivery.mp.microsoft.com

If successful, you will receive an HTTP 200 response.

Make an HTTP request from any device that the MCC server will serve content to:

http://[MCC-IP-address]/filestreamingservice/files/7bc846e0-af9c-49be-a03d-bb04428c9bb5/Microsoft.png?cacheHostOrigin=dl.delivery.mp.microsoft.com

If successful, the device will download a PNG from the cache server.

Back in the Azure portal, the cache node will show a status of 'Healthy' and show the currently installed version.

Performance metrics will be visible in the 'Overview' tab. Detailed metrics can be viewed in the 'Metrics' tab. The charts will be blank until devices are configured to use the MCC server and perform download activities.

Next steps​

The MCC server is now ready to servce cached Microsoft update content to any client that asks! Stay tuned for the next post on how to configure devices to use the MCC server effectively.

Resources​

AZ Commands​

The AZ CLI supports creation of Connected Cache resources and nodes with the following commands. These commands are in preview, so stay up to date with the documentation!

Create Connected Cache Azure resource. https://learn.microsoft.com/en-us/cli/azure/mcc/ent/resource

$subscription = "Production"
$mccResourceName = "ConnCacheBranches"
$resourceLocation = "westus"
$resourceGroup = "ConnCacheBranches"
az account set --subscription $subscription
az mcc ent resource create --mcc-resource-name $mccResourceName --location $resourceLocation --resource-group $resourceGroup

Create Connected Cache node. https://learn.microsoft.com/en-us/cli/azure/mcc/ent/node

$subscription = "Production"
$mccResourceName = "ConnCacheBranches"
$resourceLocation = "westus"
$resourceGroup = "ConnCacheBranches"

$cacheNodeName = "Office1"
$cacheNodeOperatingSystem = "Windows"
$driveSizeGB = "100"

$proxy = "enabled"
$proxyHost = "proxy.company.com"
$proxyPort = "80"

$autoUpdateRing = "slow"
$autoUpdateDay = "6" #Saturday
$autoUpdateTime = "00:00" #midnight UTC.
$autoUpdateWeek = "4"

az account set --subscription $subscription
az mcc ent node create --cache-node-name $name --mcc-resource-name $mccResourceName --host-os $cacheNodeOperatingSystem --resource-group $resourceGroup
az mcc ent node wait --created --cache-node-name $name --mcc-resource-name $mccResourceName --resource-group $resourceGroup
#Configure cache node
az mcc ent node update --cache-node-name $name --mcc-resource-name $mccResourceName --resource-group $resourceGroup --cache-drive "[{physical-path:/var/mcc,size-in-gb:$driveSizeGB}]" --proxy $proxy --proxy-host $proxyHost --proxy-port $proxyPort --auto-update-ring $autoUpdateRing --auto-update-day $autoUpdateDay --auto-update-time $autoUpdateTime --auto-update-week $autoUpdateWeek