This is a log of stuff I did w/ one of my random pet projects, it’s not terribly interesting and is mostly just so I can get into the habit of writing more. You have been warned.

(Note to self: I should really revise these, even if they’re more like stream of consciousness than organized, intentional, writing.)

So last time I had just implemented the Full Text Search index for every recipe. There was some new code to populate the index and query it, allowing multiple recipe databases to be queried as one. Now, after, several days of fucking around, I have a working recipe search:

Mealplanner

(Do note the theme is very much not final :) ).

The work of hooking up the Vue user interface to the recipe search code was quick. Getting it to work was where things got interesting.

Getting the FTS query to work

I wrote the code to search the recipe index last time, but only got existing tests to work, and the existing tests just checked if fetching every recipe worked. There was no test for a specific search term, so of course that didn’t work at first.

The original code I wrote had used a SQL function for FTS called OFFSETS() to try and provide better search results. I wanted results that had occurrences in certain fields to have precendence over others. For instance, if a search term appears in a title or in the ingredient list, it should appear before a recipe that contained the search term only in the description of a specific step.

OFFSETS(), which returns the offsets of the fields that were matched (among other things), let me do this by ordering by the first character in the result of the function. In other words, since the title field was first and the recipe_text field second, if the search term was in the title, the offset character would be 0. If it was in recipe_text alone, it would be 1.

But this function did not exist in the version of FTS I was using (FTS5). So I switched to FTS4 and tried using it, only to find out another feature, the existing search result ranking implementation is only in FTS5. I really didn’t want to have to write my own SQLite extension to either implement my own ranking function or OFFSETS() function, so I just got rid of the use of OFFSETS() and added a note to check how useful the search results are without it.

Once that was done (and I wrote some missing tests), the recipe search started working, except for one thing: the images in the search results were not displaying.

Reducing the size of the plated recipe database

The original code I had in the plated import for importing images was very ad-hoc. I know it worked, I had images displaying a while earlier, before I put in SQLite persistence, but it was very… hard to trust. It was also rather space inefficient, basically just copying raw pixel data and storing it in SQLite.

I didn’t really know where to start looking for the reason images weren’t displaying, but this seemed like a good place to start. Especially since I also needed to whittle down the size of the imported plated recipe database.

I looked at a couple JavaScript solutions for image loading and manipulation, and tried a couple of them (because I am still not comfortable enough w/ rust to actually want to use it). But none of them panned out. And of course, since all JavaScript code is single threaded, the long execution times for image loading/manipulation meant the app would just lock while importing.

So I settled for using rust’s image crate in a new Tauri plugin: https://gitlab.com/dizzycodes/tauri-plugin-image/. I thought about implementing the entire image import logic in one rust function that would be exposed, but… the entire point of me using tauri was to NOT have to write native code for desktop/mobile app development.

So instead, I created very lightweight bindings for the image crate (though just the functionality I needed right now). Essentially, primary functionality of image, like the from_vec function, image resizing and JPEG encoding. This results in minimal rust code I have to write/maintain, keeping the main app code in JavaScript (which is far easier for me to write and test).

Anyway, did that, added some image resizing during the import and saved the images to JPEGs in SQLite to reduce the size of the plated database.

This reduced the total size of the imported plated database from 1.5 GB to 260 MB. Which is a massive improvement, but it’s still a bit bigger than I’d want it to be. I may revisit this again later.

And yet images still weren’t loading.

Getting images to load properly

This was rather time consuming to pin down. I added some debugging code to save the images in tauri-plugin-image to see what the actual image data looked like. And it looked like static. So at some point the image data was being corrupted.

The difficulty here, was that I had no idea how well written my rust code was. I’m pretty unfamiliar w/ the memory model, it was very possible I was doing something wrong and, I don’t know, reading memory I shouldn’t be?

Turns out it had nothing to do w/ tauri-plugin-image. Well not really, there were some issues in how I was using the rust image crate. But the primary issue is in how I was trying to send binary data from JavaScript to rust.

The binary image data in JS is stored in TypedArrays (what you’d expect for binary data), but these arrays were not being correctly passed to rust code (in fact, the process would just hang if I tried to send these by itself). I originally tried getting around this by encoding the binary data in a string, and sending that.

In JavaScript it was possible to temporarily store binary data in a string and get the same data out. This did not work when sending the data to rust, something corrupted it. Digging through the tauri code, I found it was due to tauri sending data to rust as JSON (via JSON.stringify), which does not support binary data.

To get things to work I started converting the typed arrays to normal JS arrays and sending those. This could be read in rust as a Vec<u8> and the data was not modified in any way. Though it is a bit memory inefficient I think. I’ll switch to base64 eventually.

And this is where I stopped working, with a very minimally functional app, with one working feature: searching for recipes in a recipe database.

Might not sound like much, but it’s the real starting point for me. From here, I can add new features with confidence since most of the things I was unsure of now have a proven solution :). Next on my todo list is creating a prioritized roadmap of features I can work on one by one. Then just working on them :).