Jekyll2023-08-31T15:05:48+00:00http://baylorrae.com/feed.xmlBaylor Rae’I'm an ambitious web developer and designer who loves learning better programming practices and awesome design techniques.
Create a Windows 10 USB from MacOS2021-02-12T13:00:00+00:002021-02-12T13:00:00+00:00http://baylorrae.com/create-windows-10-usb-on-macos<p>These are the steps I use to create a Window 10 installer on MacOS. I only do this once in a blue moon and I want to make sure I keep track of these commands before they are lost in my history.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Format the USB drive as MS-DOS and name it "WIN10"</span>
<span class="nv">$ </span>diskutil eraseDisk MS-DOS <span class="s2">"WIN10"</span> MBR /dev/disk127
<span class="c"># Copy files from mounted Windows 10 ISO to new drive</span>
<span class="c"># > Make sure to preserve trailing slash (/)</span>
<span class="nv">$ </span>rsync <span class="nt">-vha</span> <span class="nt">--exclude</span><span class="o">=</span>sources/install.wim /Volumes/CCCOMA_X64FRE_EN-US_DV9/ /Volumes/WIN10
<span class="c"># Split and copy the last file</span>
<span class="nv">$ </span>wimlib-imagex <span class="nb">split</span> /Volumes/CCCOMA_X64FRE_EN-US_DV9/sources/install.wim /Volumes/WIN10/sources/install.wim 4000
</code></pre></div></div>
<p>The prerequisites for the above commands are knowing the id of the mounted USB drive and <code class="language-plaintext highlighter-rouge">wimlib-imagex</code>. MacOS only has write capabilities for the older <code class="language-plaintext highlighter-rouge">MS-DOS</code> file format which has a 4GB single file size limit. This is why we use <code class="language-plaintext highlighter-rouge">wimlib-imagex</code> to split the 5GB install.wim into two files.</p>
<p>You can find the ID of your USB drive with Disk Utility.</p>
<p><img src="assets/disk-utility-windows-usb.png" alt="Disk Utility" /></p>
<p>It looks like I installed <code class="language-plaintext highlighter-rouge">wimlib-imagex</code> with <a href="https://brew.sh/">HomeBrew</a>.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>brew <span class="nb">install </span>wimlib
</code></pre></div></div>
<p>See you later future Baylor!</p>These are the steps I use to create a Window 10 installer on MacOS. I only do this once in a blue moon and I want to make sure I keep track of these commands before they are lost in my history.Wrangle Your SQL With Arel by Eric Hayes2020-09-26T13:00:00+00:002020-09-26T13:00:00+00:00http://baylorrae.com/wrangle-your-sql-with-arel<p>This is a fascinating talk that walks through the underlying APIs used to build SQL queries in ActiveRecord. Eric does a great job showing how ORMs use Abstract Syntax Trees to create SQL queries from hashes and method calls.</p>
<p>The end result shows how they built an incredibly complex cohort query with efficiency and maintainablity using Arel.</p>
<div class="embed-container">
<iframe src="https://www.youtube.com/embed/8pQGzsDzYEo" width="700" height="480" frameborder="0" allowfullscreen="true">
</iframe>
</div>This is a fascinating talk that walks through the underlying APIs used to build SQL queries in ActiveRecord. Eric does a great job showing how ORMs use Abstract Syntax Trees to create SQL queries from hashes and method calls.Process HEIC Images with Node on AWS Lambda2020-01-13T13:00:00+00:002020-01-13T13:00:00+00:00http://baylorrae.com/aws-lambda-heic-and-sharp<p>I needed to add support for HEIF/HEIC images in my application and thought it
would be a fun introduction to AWS Lambda. Since I have minimal experience with
AWS and this was an exercise for myself I thought it would also be fun to use
<a href="https://sharp.pixelplumbing.com/en/stable/">Sharp</a>.</p>
<p>The first thing I learned is that HEIF is not shipped with image processing
libraries, ImageMagick or Sharp. From what I can tell this is mostly because of
licensing issues that thankfully don’t apply to me. As a result, the first
requirement is to compile a <a href="https://github.com/libvips/libvips">Vips</a> with a
self install version of <a href="https://github.com/strukturag/libheif">libheif</a> and
<a href="https://github.com/strukturag/libde265">libde265</a>.</p>
<p>This is the largest hurdle since AWS Lambda runs application code in a
stateless, isolated environment. To overcome this we need to build and package
these libraries and attach them to our Lambda function. I use the word “attach”
since I uploaded these binaries as a Lambda layer to lighten my application
code.</p>
<p>I used a <a href="#dockerfile">multi-stage Dockerfile</a> to build the binaries, copy them
into another isolated environment and install the sharp node module. I then
wrapped these steps in <a href="#compile-and-copy-script">a shell script</a> to further the
portability of it.</p>
<h3 id="dockerfile">Dockerfile</h3>
<div class="language-docker highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Use environment that closely matches AWS Lambda</span>
<span class="k">FROM</span><span class="s"> lambci/lambda:build-nodejs12.x AS builder</span>
<span class="k">WORKDIR</span><span class="s"> /var/task</span>
<span class="c"># Download dependencies for vips, libheif and libde265</span>
<span class="k">RUN </span>curl <span class="nt">-L</span> https://github.com/libvips/libvips/releases/download/v8.9.0/vips-8.9.0.tar.gz | <span class="nb">tar </span>xz <span class="o">&&</span> <span class="se">\
</span> curl <span class="nt">-L</span> https://github.com/strukturag/libde265/releases/download/v1.0.4/libde265-1.0.4.tar.gz | <span class="nb">tar </span>xz <span class="o">&&</span> <span class="se">\
</span> curl <span class="nt">-L</span> https://github.com/strukturag/libheif/releases/download/v1.6.1/libheif-1.6.1.tar.gz | <span class="nb">tar </span>xz <span class="o">&&</span> <span class="se">\
</span> yum <span class="nb">install</span> <span class="nt">-y</span> gtk-doc gobject-introspection gobject-introspection-devel expat-devel libjpeg-turbo libjpeg-turbo-devel libpng libpng-devel
<span class="c"># Install h.265 video codec library</span>
<span class="k">RUN </span><span class="nb">cd </span>libde265-1.0.4 <span class="o">&&</span> <span class="se">\
</span> ./autogen.sh <span class="o">&&</span> <span class="se">\
</span> ./configure <span class="o">&&</span> <span class="se">\
</span> make <span class="o">&&</span> <span class="se">\
</span> make <span class="nb">install</span>
<span class="c"># Install HEIF library</span>
<span class="k">RUN </span><span class="nb">cd </span>libheif-1.6.1 <span class="o">&&</span> <span class="se">\
</span> ./autogen.sh <span class="o">&&</span> <span class="se">\
</span> <span class="nv">PKG_CONFIG_PATH</span><span class="o">=</span>/usr/local/lib/pkgconfig ./configure <span class="o">&&</span> <span class="se">\
</span> make <span class="o">&&</span> <span class="se">\
</span> make <span class="nb">install</span>
<span class="c"># Install vips</span>
<span class="k">RUN </span><span class="nb">cd </span>vips-8.9.0 <span class="o">&&</span> <span class="se">\
</span> ./autogen.sh <span class="o">&&</span> <span class="se">\
</span> <span class="nv">PKG_CONFIG_PATH</span><span class="o">=</span>/usr/local/lib/pkgconfig ./configure <span class="nt">--prefix</span><span class="o">=</span>/var/task/vendor <span class="o">&&</span> <span class="se">\
</span> make <span class="o">&&</span> <span class="se">\
</span> make <span class="nb">install</span>
<span class="c"># Copy pkg-config files to working directory</span>
<span class="k">RUN </span><span class="nb">cp</span> /usr/local/lib/pkgconfig/<span class="k">*</span> vendor/lib/pkgconfig/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/pkgconfig/libpcre<span class="k">*</span> vendor/lib/pkgconfig/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/pkgconfig/glib<span class="k">*</span> vendor/lib/pkgconfig/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/pkgconfig/gobject<span class="k">*</span> vendor/lib/pkgconfig/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/pkgconfig/gmodule<span class="k">*</span> vendor/lib/pkgconfig/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/pkgconfig/gthread<span class="k">*</span> vendor/lib/pkgconfig/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/pkgconfig/libpng<span class="k">*</span> vendor/lib/pkgconfig/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/pkgconfig/libjpeg<span class="k">*</span> vendor/lib/pkgconfig/
<span class="c"># Copy compiled binaries to working directory</span>
<span class="k">RUN </span><span class="nb">cp</span> /usr/local/lib/libheif.<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/local/lib/libde265.<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/libpng<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/libjpeg<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> <span class="nt">-r</span> /usr/lib64/glib<span class="k">*</span>/include/<span class="k">*</span> vendor/include/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> <span class="nt">-r</span> /usr/include/glib<span class="k">*</span>/<span class="k">*</span> vendor/include/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> <span class="nt">-r</span> /usr/lib64/gobject<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/libexpat<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/libgthread<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/libgmodule<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/libpcre<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/libgobject<span class="k">*</span> vendor/lib/ <span class="o">&&</span> <span class="se">\
</span> <span class="nb">cp</span> /usr/lib64/libglib<span class="k">*</span> vendor/lib/
<span class="c"># Starting in a new image ensures we have everything for</span>
<span class="c"># an isolated environment.</span>
<span class="k">FROM</span><span class="s"> lambci/lambda:build-nodejs12.x</span>
<span class="k">WORKDIR</span><span class="s"> /var/task</span>
<span class="c"># Copy results from builder image</span>
<span class="k">COPY</span><span class="s"> --from=builder /var/task/vendor ./vendor</span>
<span class="c"># Install sharp with node</span>
<span class="k">RUN </span><span class="nv">LD_LIBRARY_PATH</span><span class="o">=</span>/var/task/vendor/include <span class="nv">PKG_CONFIG_PATH</span><span class="o">=</span>/var/task/vendor/lib/pkgconfig npm <span class="nb">install </span>sharp
</code></pre></div></div>
<h3 id="compile-and-copy-script">Compile and Copy Script</h3>
<p>The following script builds and runs the docker image and copies the necessary
files to the host machine. It’s important to note that the files are copied to a
directory structure that is matches <code class="language-plaintext highlighter-rouge">$PATH</code>, <code class="language-plaintext highlighter-rouge">$NODE_PATH</code> and <code class="language-plaintext highlighter-rouge">$LD_LIBRARY_PATH</code>
inside the Lambda environment.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/usr/bin/env bash</span>
main<span class="o">()</span> <span class="o">{</span>
<span class="nb">echo</span> <span class="s2">"Building Image"</span>
<span class="nv">IMAGE_ID</span><span class="o">=</span><span class="si">$(</span>docker build <span class="nb">.</span> <span class="nt">-q</span><span class="si">)</span>
<span class="nb">echo</span> <span class="s2">"Starting Container"</span>
<span class="nv">CONTAINER_ID</span><span class="o">=</span><span class="si">$(</span>docker run <span class="nt">-d</span> <span class="nt">-it</span> <span class="nv">$IMAGE_ID</span> bash<span class="si">)</span>
<span class="nb">echo</span> <span class="s2">"Copying node_modules into node_modules"</span>
docker <span class="nb">cp</span> <span class="nv">$CONTAINER_ID</span>:/var/task/node_modules ./vips-sharp-layer-env/nodejs
<span class="nb">echo</span> <span class="s2">"Copying vendor into vendor"</span>
docker <span class="nb">cp</span> <span class="nv">$CONTAINER_ID</span>:/var/task/vendor/bin ./vips-sharp-layer-env
docker <span class="nb">cp</span> <span class="nv">$CONTAINER_ID</span>:/var/task/vendor/lib ./vips-sharp-layer-env
<span class="nb">echo</span> <span class="s2">"Removing container"</span>
docker <span class="nb">kill</span> <span class="nv">$CONTAINER_ID</span>
docker <span class="nb">rm</span> <span class="nv">$CONTAINER_ID</span>
<span class="o">}</span>
main
</code></pre></div></div>
<h3 id="creating-the-lambda-layer">Creating the Lambda Layer</h3>
<p>I’m using CloudFormation and the CDK to build out my application. The following
will upload the results from the above script as a Layer and add it to a Lambda
Function.</p>
<div class="language-ts highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">uploadBucket</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">s3</span><span class="p">.</span><span class="nx">Bucket</span><span class="p">(</span><span class="k">this</span><span class="p">,</span> <span class="dl">"</span><span class="s2">Upload</span><span class="dl">"</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">outputBucket</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">s3</span><span class="p">.</span><span class="nx">Bucket</span><span class="p">(</span>
<span class="k">this</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">Output</span><span class="dl">"</span><span class="p">,</span>
<span class="p">{</span>
<span class="na">blockPublicAccess</span><span class="p">:</span> <span class="nx">BlockPublicAccess</span><span class="p">.</span><span class="nx">BLOCK_ACLS</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">)</span>
<span class="nx">outputBucket</span><span class="p">.</span><span class="nx">addToResourcePolicy</span><span class="p">(</span><span class="k">new</span> <span class="nx">iam</span><span class="p">.</span><span class="nx">PolicyStatement</span><span class="p">({</span>
<span class="na">effect</span><span class="p">:</span> <span class="nx">iam</span><span class="p">.</span><span class="nx">Effect</span><span class="p">.</span><span class="nx">ALLOW</span><span class="p">,</span>
<span class="na">principals</span><span class="p">:</span> <span class="p">[</span><span class="k">new</span> <span class="nx">iam</span><span class="p">.</span><span class="nx">AnyPrincipal</span><span class="p">()],</span>
<span class="na">actions</span><span class="p">:</span> <span class="p">[</span><span class="dl">'</span><span class="s1">s3:GetObject</span><span class="dl">'</span><span class="p">],</span>
<span class="na">resources</span><span class="p">:</span> <span class="p">[</span><span class="s2">`arn:aws:s3:::</span><span class="p">${</span><span class="nx">outputBucket</span><span class="p">.</span><span class="nx">bucketName</span><span class="p">}</span><span class="s2">/*`</span><span class="p">]</span>
<span class="p">}))</span>
<span class="kd">const</span> <span class="nx">vipsSharpLayer</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">lambda</span><span class="p">.</span><span class="nx">LayerVersion</span><span class="p">(</span>
<span class="k">this</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">VipsSharpLayer</span><span class="dl">"</span><span class="p">,</span>
<span class="p">{</span>
<span class="na">compatibleRuntimes</span><span class="p">:</span> <span class="p">[</span><span class="nx">lambda</span><span class="p">.</span><span class="nx">Runtime</span><span class="p">.</span><span class="nx">NODEJS_12_X</span><span class="p">],</span>
<span class="na">code</span><span class="p">:</span> <span class="nx">lambda</span><span class="p">.</span><span class="nx">Code</span><span class="p">.</span><span class="nx">fromAsset</span><span class="p">(</span><span class="dl">"</span><span class="s2">lambda/process-images/vips-sharp-layer-env</span><span class="dl">"</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">)</span>
<span class="kd">const</span> <span class="nx">resizeHandler</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">lambda</span><span class="p">.</span><span class="nb">Function</span><span class="p">(</span>
<span class="k">this</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">ResizeHandler</span><span class="dl">"</span><span class="p">,</span>
<span class="p">{</span>
<span class="na">runtime</span><span class="p">:</span> <span class="nx">lambda</span><span class="p">.</span><span class="nx">Runtime</span><span class="p">.</span><span class="nx">NODEJS_12_X</span><span class="p">,</span>
<span class="na">code</span><span class="p">:</span> <span class="nx">lambda</span><span class="p">.</span><span class="nx">Code</span><span class="p">.</span><span class="nx">fromAsset</span><span class="p">(</span><span class="dl">"</span><span class="s2">lambda/process-images/process-images</span><span class="dl">"</span><span class="p">),</span>
<span class="na">handler</span><span class="p">:</span> <span class="dl">"</span><span class="s2">resize.handler</span><span class="dl">"</span><span class="p">,</span>
<span class="na">timeout</span><span class="p">:</span> <span class="nx">Duration</span><span class="p">.</span><span class="nx">seconds</span><span class="p">(</span><span class="mi">30</span><span class="p">),</span>
<span class="na">layers</span><span class="p">:</span> <span class="p">[</span><span class="nx">vipsSharpLayer</span><span class="p">],</span>
<span class="na">environment</span><span class="p">:</span> <span class="p">{</span>
<span class="na">OUTPUT_BUCKET</span><span class="p">:</span> <span class="nx">outputBucket</span><span class="p">.</span><span class="nx">bucketName</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">)</span>
<span class="nx">uploadBucket</span><span class="p">.</span><span class="nx">grantRead</span><span class="p">(</span><span class="nx">resizeHandler</span><span class="p">)</span>
<span class="nx">outputBucket</span><span class="p">.</span><span class="nx">grantWrite</span><span class="p">(</span><span class="nx">resizeHandler</span><span class="p">)</span>
<span class="nx">resizeHandler</span><span class="p">.</span><span class="nx">addEventSource</span><span class="p">(</span><span class="k">new</span> <span class="nx">S3EventSource</span><span class="p">(</span><span class="nx">uploadBucket</span><span class="p">,</span> <span class="p">{</span>
<span class="na">events</span><span class="p">:</span> <span class="p">[</span><span class="nx">s3</span><span class="p">.</span><span class="nx">EventType</span><span class="p">.</span><span class="nx">OBJECT_CREATED</span><span class="p">],</span>
<span class="p">}))</span>
</code></pre></div></div>
<h3 id="the-lambda-function">The Lambda Function</h3>
<p>This code barely scratches the surface of what I want it to do long term. Right
now it simply loads the S3 object into sharp, formats it to JPG and uploads it
into an output bucket.</p>
<p>I’ve also introduced <a href="https://github.com/mattiasw/ExifReader">ExifReader</a> to
read the orientation of the image to ensure it is rotated correctly during the
upload. Sharp has support for auto rotating the image but I found it couldn’t
read the orientation data from HEIC images. This is unfortunate as it requires
me to load the entire image into memory rather than stream it into the output
bucket.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">AWS</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">aws-sdk</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">sharp</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">sharp</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">ExifReader</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">exifreader</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">S3</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">AWS</span><span class="p">.</span><span class="nx">S3</span><span class="p">({</span>
<span class="na">maxRetries</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
<span class="na">region</span><span class="p">:</span> <span class="dl">'</span><span class="s1">us-east-1</span><span class="dl">'</span>
<span class="p">})</span>
<span class="kd">const</span> <span class="nx">ORIENTATION_MAP</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">auto</span><span class="p">:</span> <span class="kc">undefined</span><span class="p">,</span>
<span class="mi">3</span><span class="p">:</span> <span class="mi">180</span><span class="p">,</span>
<span class="mi">6</span><span class="p">:</span> <span class="mi">90</span><span class="p">,</span>
<span class="mi">8</span><span class="p">:</span> <span class="o">-</span><span class="mi">90</span>
<span class="p">}</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">handler</span> <span class="o">=</span> <span class="k">async</span> <span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">context</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">srcBucket</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">Records</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">s3</span><span class="p">.</span><span class="nx">bucket</span><span class="p">.</span><span class="nx">name</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">srcKey</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">Records</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nx">s3</span><span class="p">.</span><span class="nx">object</span><span class="p">.</span><span class="nx">key</span><span class="p">;</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">`Getting </span><span class="p">${</span><span class="nx">srcKey</span><span class="p">}</span><span class="s2"> from s3://</span><span class="p">${</span><span class="nx">srcBucket</span><span class="p">}</span><span class="s2">`</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">imageObject</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">S3</span><span class="p">.</span><span class="nx">getObject</span><span class="p">({</span>
<span class="na">Bucket</span><span class="p">:</span> <span class="nx">srcBucket</span><span class="p">,</span>
<span class="na">Key</span><span class="p">:</span> <span class="nx">srcKey</span>
<span class="p">}).</span><span class="nx">promise</span><span class="p">()</span>
<span class="kd">const</span> <span class="nx">image</span> <span class="o">=</span> <span class="nx">imageObject</span><span class="p">.</span><span class="nx">Body</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">Read exif data</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">let</span> <span class="nx">orientation</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">auto</span><span class="dl">'</span>
<span class="k">try</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">exif</span> <span class="o">=</span> <span class="nx">ExifReader</span><span class="p">.</span><span class="nx">load</span><span class="p">(</span><span class="nx">image</span><span class="p">)</span>
<span class="nx">orientation</span> <span class="o">=</span> <span class="nx">exif</span> <span class="o">&&</span> <span class="nx">exif</span><span class="p">.</span><span class="nx">Orientation</span> <span class="o">&&</span> <span class="nx">exif</span><span class="p">.</span><span class="nx">Orientation</span><span class="p">.</span><span class="nx">value</span> <span class="o">||</span> <span class="dl">'</span><span class="s1">auto</span><span class="dl">'</span>
<span class="p">}</span><span class="k">catch</span> <span class="p">(</span><span class="nx">e</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">Failed reading exif data</span><span class="dl">'</span><span class="p">)</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">e</span><span class="p">)</span>
<span class="p">}</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">Building Sharp Pipeline </span><span class="se">\'</span><span class="s1">toJPG</span><span class="se">\'</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">output</span> <span class="o">=</span> <span class="nx">sharp</span><span class="p">(</span><span class="nx">image</span><span class="p">)</span>
<span class="p">.</span><span class="nx">rotate</span><span class="p">(</span><span class="nx">ORIENTATION_MAP</span><span class="p">[</span><span class="nx">orientation</span><span class="p">])</span>
<span class="p">.</span><span class="nx">toFormat</span><span class="p">(</span><span class="dl">'</span><span class="s1">jpg</span><span class="dl">'</span><span class="p">)</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">Upload image</span><span class="dl">'</span><span class="p">)</span>
<span class="kd">const</span> <span class="nx">uploadData</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">S3</span><span class="p">.</span><span class="nx">upload</span><span class="p">({</span>
<span class="na">Body</span><span class="p">:</span> <span class="k">await</span> <span class="nx">output</span><span class="p">.</span><span class="nx">toBuffer</span><span class="p">(),</span>
<span class="na">Bucket</span><span class="p">:</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">OUTPUT_BUCKET</span><span class="p">,</span>
<span class="na">ContentType</span><span class="p">:</span> <span class="dl">'</span><span class="s1">image/jpg</span><span class="dl">'</span><span class="p">,</span>
<span class="na">Key</span><span class="p">:</span> <span class="nx">srcKey</span><span class="p">.</span><span class="nx">split</span><span class="p">(</span><span class="dl">'</span><span class="s1">.</span><span class="dl">'</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="dl">'</span><span class="s1">.jpg</span><span class="dl">'</span>
<span class="p">}).</span><span class="nx">promise</span><span class="p">()</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">Data: </span><span class="dl">'</span><span class="p">,</span> <span class="p">{</span>
<span class="p">...</span><span class="nx">uploadData</span>
<span class="p">})</span>
<span class="k">return</span> <span class="p">{</span>
<span class="na">message</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Success</span><span class="dl">'</span><span class="p">,</span>
<span class="na">timeRemainingMs</span><span class="p">:</span> <span class="nx">context</span><span class="p">.</span><span class="nx">getRemainingTimeInMillis</span><span class="p">()</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>And that’s it. Aside from locating all the necessary binaries to create a
portable version of Vips it is pretty straight forward. Hopefully you can use
this as a reference for your own project and you findit helpful.</p>I needed to add support for HEIF/HEIC images in my application and thought it would be a fun introduction to AWS Lambda. Since I have minimal experience with AWS and this was an exercise for myself I thought it would also be fun to use Sharp.Basecamp’s Trix with React2019-11-06T13:00:00+00:002019-11-06T13:00:00+00:00http://baylorrae.com/basecamp-trix-with-react<p>I needed to add the Trix Editor to my React application and couldn’t find a
simple solution. Since having an actual <code class="language-plaintext highlighter-rouge"><input /></code> node in the DOM isn’t
required, I opted to have the value bound directly the editor’s built in input
node.</p>
<h3 id="usage">Usage</h3>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="nx">React</span><span class="p">,</span> <span class="p">{</span> <span class="nx">useState</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">Form</span> <span class="o">=</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="p">[</span><span class="nx">body</span><span class="p">,</span> <span class="nx">setBody</span><span class="p">]</span> <span class="o">=</span> <span class="nx">useState</span><span class="p">(</span><span class="dl">''</span><span class="p">)</span>
<span class="k">return</span> <span class="p">(</span>
<span class="o"><</span><span class="nx">Editor</span> <span class="nx">value</span><span class="o">=</span><span class="p">{</span><span class="nx">body</span><span class="p">}</span> <span class="nx">onChange</span><span class="o">=</span><span class="p">{</span><span class="nx">setBody</span><span class="p">}</span> <span class="sr">/</span><span class="err">>
</span> <span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="the-editor-component">The <code class="language-plaintext highlighter-rouge">Editor</code> Component</h3>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="nx">React</span><span class="p">,</span> <span class="p">{</span> <span class="nx">useEffect</span><span class="p">,</span> <span class="nx">useRef</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">react</span><span class="dl">'</span>
<span class="k">import</span> <span class="dl">'</span><span class="s1">trix</span><span class="dl">'</span>
<span class="k">import</span> <span class="dl">'</span><span class="s1">trix/dist/trix.css</span><span class="dl">'</span>
<span class="kd">const</span> <span class="nx">Editor</span> <span class="o">=</span> <span class="p">({</span> <span class="nx">value</span><span class="p">,</span> <span class="nx">onChange</span> <span class="p">})</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">trixEditor</span> <span class="o">=</span> <span class="nx">useRef</span><span class="p">(</span><span class="kc">null</span><span class="p">)</span>
<span class="nx">useEffect</span><span class="p">(()</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">trixEditor</span><span class="p">.</span><span class="nx">current</span><span class="p">.</span><span class="nx">addEventListener</span><span class="p">(</span><span class="dl">'</span><span class="s1">trix-change</span><span class="dl">'</span><span class="p">,</span> <span class="p">(</span><span class="nx">e</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">onChange</span><span class="p">(</span><span class="nx">trixEditor</span><span class="p">.</span><span class="nx">current</span><span class="p">.</span><span class="nx">value</span><span class="p">)</span>
<span class="p">})</span>
<span class="p">},</span> <span class="p">[</span><span class="nx">trixEditor</span><span class="p">])</span>
<span class="nx">useEffect</span><span class="p">(()</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">trixEditor</span><span class="p">.</span><span class="nx">current</span><span class="p">)</span> <span class="k">return</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">trixEditor</span><span class="p">.</span><span class="nx">current</span><span class="p">.</span><span class="nx">inputElement</span><span class="p">.</span><span class="nx">value</span> <span class="o">===</span> <span class="nx">value</span><span class="p">)</span> <span class="k">return</span>
<span class="nx">trixEditor</span><span class="p">.</span><span class="nx">current</span><span class="p">.</span><span class="nx">value</span> <span class="o">=</span> <span class="nx">value</span>
<span class="p">},</span> <span class="p">[</span><span class="nx">value</span><span class="p">])</span>
<span class="k">return</span> <span class="p">(</span>
<span class="nx">React</span><span class="p">.</span><span class="nx">createElement</span><span class="p">(</span><span class="dl">'</span><span class="s1">trix-editor</span><span class="dl">'</span><span class="p">,</span> <span class="p">{</span><span class="na">ref</span><span class="p">:</span> <span class="nx">trixEditor</span><span class="p">})</span>
<span class="p">)</span>
<span class="p">}</span>
<span class="k">export</span> <span class="k">default</span> <span class="nx">Editor</span>
</code></pre></div></div>I needed to add the Trix Editor to my React application and couldn’t find a simple solution. Since having an actual <input /> node in the DOM isn’t required, I opted to have the value bound directly the editor’s built in input node.Github Actions with Rails and Postgres2019-11-06T03:47:46+00:002019-11-06T03:47:46+00:00http://baylorrae.com/github-actions-rails-and-postgres<p>Github Actions were recently made widely available and this workflow has been
awesome. It runs Postgres in a docker container and installs the <code class="language-plaintext highlighter-rouge">libpq-dev</code>
package for the <code class="language-plaintext highlighter-rouge">pg</code> gem. Enjoy!</p>
<blockquote>
<p>It’s important to note that this configuration runs Postgres inside a docker
container with the port forwarded to a random, available port within the
build’s VM. Don’t forget to modify your <code class="language-plaintext highlighter-rouge">config/database.yml</code> to read the
configuration from the assigned environment variables.</p>
</blockquote>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">name</span><span class="pi">:</span> <span class="s">Ruby</span>
<span class="na">on</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">push</span><span class="pi">]</span>
<span class="na">jobs</span><span class="pi">:</span>
<span class="na">rspec</span><span class="pi">:</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
<span class="na">services</span><span class="pi">:</span>
<span class="na">postgres</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">postgres:10.8</span>
<span class="na">env</span><span class="pi">:</span>
<span class="na">POSTGRES_USER</span><span class="pi">:</span> <span class="s">postgres</span>
<span class="na">POSTGRES_PASSWORD</span><span class="pi">:</span> <span class="s">postgres</span>
<span class="na">POSTGRES_DB</span><span class="pi">:</span> <span class="s">RAILS_APP_DB_NAME_test</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="c1"># will assign a random free host port</span>
<span class="pi">-</span> <span class="s">5432/tcp</span>
<span class="c1"># needed because the postgres container does not provide a healthcheck</span>
<span class="na">options</span><span class="pi">:</span> <span class="s">--health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries </span><span class="m">5</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v1</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Install Postgres Libraries</span>
<span class="na">run</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">sudo apt-get install libpq-dev</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Set up Ruby </span><span class="m">2.6</span>
<span class="na">uses</span><span class="pi">:</span> <span class="s">actions/setup-ruby@v1</span>
<span class="na">with</span><span class="pi">:</span>
<span class="na">ruby-version</span><span class="pi">:</span> <span class="s">2.6.x</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Bundle Install</span>
<span class="na">run</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">gem install bundler</span>
<span class="s">bundle install --jobs 4 --retry 3</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Build and test with RSpec</span>
<span class="na">env</span><span class="pi">:</span>
<span class="na">POSTGRES_HOST</span><span class="pi">:</span> <span class="s">localhost</span>
<span class="na">POSTGRES_PORT</span><span class="pi">:</span> <span class="s">${{ job.services.postgres.ports[5432] }}</span> <span class="c1"># get randomly assigned published port</span>
<span class="na">POSTGRES_USER</span><span class="pi">:</span> <span class="s">postgres</span>
<span class="na">POSTGRES_PASS</span><span class="pi">:</span> <span class="s">postgres</span>
<span class="na">run</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">bundle exec rspec</span>
</code></pre></div></div>
<p>I’m using <code class="language-plaintext highlighter-rouge">#fetch</code> to have sensible defaults if the environment variables aren’t
defined.</p>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">test</span><span class="pi">:</span>
<span class="s"><<</span><span class="pi">:</span> <span class="nv">*default</span>
<span class="na">database</span><span class="pi">:</span> <span class="s">RAILS_APP_DB_NAME_test</span>
<span class="na">username</span><span class="pi">:</span> <span class="s"><%= ENV.fetch('POSTGRES_USER', '') %></span>
<span class="na">password</span><span class="pi">:</span> <span class="s"><%= ENV.fetch('POSTGRES_PASS', '') %></span>
<span class="na">host</span><span class="pi">:</span> <span class="s"><%= ENV.fetch('POSTGRES_HOST', 'localhost') %></span>
<span class="na">port</span><span class="pi">:</span> <span class="s"><%= ENV.fetch('POSTGRES_PORT', 5432) %></span>
</code></pre></div></div>Github Actions were recently made widely available and this workflow has been awesome. It runs Postgres in a docker container and installs the libpq-dev package for the pg gem. Enjoy!Active Link To Helper2019-08-28T15:19:46+00:002019-08-28T15:19:46+00:00http://baylorrae.com/active_link_to_helper<p>I’m often looking for a flexible solution for determining if I’m on the current
route. I say flexible because sometimes a link is bound to a single controller,
or sometimes multiple controllers, or multiple controllers with specific
actions.</p>
<ul>
<li><a href="#helper">Rails Helper</a></li>
<li><a href="#matcher">Matcher Class</a></li>
<li><a href="#usage">Usage</a></li>
<li><a href="#spec">Spec</a></li>
</ul>
<h3 id="helper">Rails Helper</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/helpers/active_link_helper.rb</span>
<span class="c1"># frozen_string_literal: true</span>
<span class="nb">require_relative</span> <span class="s2">"../../lib/active_link_path"</span>
<span class="k">module</span> <span class="nn">ActiveLinkHelper</span>
<span class="k">def</span> <span class="nf">nav_link_to</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="ss">matches: </span><span class="p">[],</span> <span class="o">&</span><span class="n">block</span><span class="p">)</span>
<span class="n">matcher</span> <span class="o">=</span> <span class="no">ActiveLinkPath</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
<span class="n">active</span> <span class="o">=</span> <span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">(</span><span class="n">matches</span><span class="p">)</span> <span class="p">?</span> <span class="s1">'active'</span> <span class="p">:</span> <span class="kp">nil</span>
<span class="n">link_to</span> <span class="n">path</span><span class="p">,</span> <span class="ss">class: </span><span class="p">[</span><span class="s1">'nav-link'</span><span class="p">,</span> <span class="n">active</span><span class="p">].</span><span class="nf">compact</span><span class="p">,</span> <span class="o">&</span><span class="n">block</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="matcher">Matcher Class</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/active_link_path.rb</span>
<span class="k">class</span> <span class="nc">ActiveLinkPath</span>
<span class="nb">attr_reader</span> <span class="ss">:context</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">context</span><span class="p">)</span>
<span class="vi">@context</span> <span class="o">=</span> <span class="n">context</span><span class="p">.</span><span class="nf">stringify_keys</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">active_match?</span><span class="p">(</span><span class="n">match</span><span class="p">)</span>
<span class="n">matches</span> <span class="o">=</span> <span class="n">match</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">Hash</span><span class="p">)</span> <span class="p">?</span> <span class="p">[</span><span class="n">match</span><span class="p">]</span> <span class="p">:</span> <span class="n">match</span>
<span class="n">matches</span><span class="p">.</span><span class="nf">any?</span> <span class="k">do</span> <span class="o">|</span><span class="n">match</span><span class="o">|</span>
<span class="n">match</span><span class="p">.</span><span class="nf">stringify_keys!</span>
<span class="n">values_match</span><span class="p">(</span><span class="n">match</span><span class="p">,</span> <span class="n">context</span><span class="p">.</span><span class="nf">slice</span><span class="p">(</span><span class="o">*</span><span class="n">match</span><span class="p">.</span><span class="nf">keys</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">values_match</span><span class="p">(</span><span class="n">match</span><span class="p">,</span> <span class="n">sliced_context</span><span class="p">)</span>
<span class="n">match</span><span class="p">.</span><span class="nf">all?</span> <span class="k">do</span> <span class="o">|</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span><span class="o">|</span>
<span class="n">value</span> <span class="o">=</span> <span class="no">Array</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="n">value</span><span class="p">.</span><span class="nf">include?</span><span class="p">(</span><span class="n">sliced_context</span><span class="p">[</span><span class="n">key</span><span class="p">])</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="usage">Usage</h3>
<p>The API is somewhat complex to accommodate most use cases.</p>
<ol>
<li>Basic resource <code class="language-plaintext highlighter-rouge">resources :products</code></li>
<li>Nested resources <code class="language-plaintext highlighter-rouge">resources :collections { resources :products }</code></li>
<li>Top level REST actions <code class="language-plaintext highlighter-rouge">get '/export/products', to: 'export#products'</code></li>
</ol>
<p>In the above list, #3 is the most interesting since you’d probably access this
action from the products index page (<code class="language-plaintext highlighter-rouge">/products</code>).</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">nav_link_to</span><span class="p">(</span><span class="n">some_path</span><span class="p">,</span>
<span class="ss">matches: </span><span class="p">{</span><span class="ss">controller: </span><span class="s1">'products'</span><span class="p">})</span>
<span class="c1"># 👍 /products products#index</span>
<span class="c1"># 👍 /products/1 products#show</span>
<span class="c1"># 👍 /products/1/edit products#edit</span>
<span class="c1"># 👍 /collections/1/products products#index</span>
<span class="c1"># ❌ /collections collections#index</span>
<span class="c1"># ❌ /collections/1 collections#show</span>
<span class="c1"># ❌ /export/products export#products</span>
</code></pre></div></div>
<hr />
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">nav_link_to</span><span class="p">(</span><span class="n">some_path</span><span class="p">,</span>
<span class="ss">matches: </span><span class="p">{</span><span class="ss">controller: </span><span class="sx">%w(products collections)</span><span class="p">})</span>
<span class="c1"># 👍 /products products#index</span>
<span class="c1"># 👍 /products/1 products#show</span>
<span class="c1"># 👍 /products/1/edit products#edit</span>
<span class="c1"># 👍 /collections/1/products products#index</span>
<span class="c1"># 👍 /collections collections#index</span>
<span class="c1"># 👍 /collections/1 collections#show</span>
<span class="c1"># ❌ /export/products export#products</span>
</code></pre></div></div>
<hr />
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">nav_link_to</span><span class="p">(</span><span class="n">some_path</span><span class="p">,</span>
<span class="ss">matches: </span><span class="p">{</span><span class="ss">controller: </span><span class="s1">'products'</span><span class="p">,</span>
<span class="ss">action: </span><span class="s1">'show'</span><span class="p">})</span>
<span class="c1"># 👍 /products/1 products#show</span>
<span class="c1"># ❌ /products products#index</span>
<span class="c1"># ❌ /products/1/edit products#edit</span>
<span class="c1"># ❌ /collections/1/products products#index</span>
<span class="c1"># ❌ /collections collections#index</span>
<span class="c1"># ❌ /collections/1 collections#show</span>
<span class="c1"># ❌ /export/products export#products</span>
</code></pre></div></div>
<hr />
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">nav_link_to</span><span class="p">(</span><span class="n">some_path</span><span class="p">,</span>
<span class="ss">matches: </span><span class="p">{</span><span class="ss">controller: </span><span class="sx">%w(products collections)</span><span class="p">,</span>
<span class="ss">action: </span><span class="s1">'show'</span><span class="p">})</span>
<span class="c1"># 👍 /products/1 products#show</span>
<span class="c1"># 👍 /collections/1 collections#show</span>
<span class="c1"># ❌ /products products#index</span>
<span class="c1"># ❌ /products/1/edit products#edit</span>
<span class="c1"># ❌ /collections/1/products products#index</span>
<span class="c1"># ❌ /collections collections#index</span>
<span class="c1"># ❌ /export/products export#products</span>
</code></pre></div></div>
<hr />
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">nav_link_to</span><span class="p">(</span><span class="n">some_path</span><span class="p">,</span>
<span class="ss">matches: </span><span class="p">[</span>
<span class="p">{</span><span class="ss">controller: </span><span class="sx">%w(products collections)</span><span class="p">,</span> <span class="ss">action: </span><span class="s1">'index'</span><span class="p">},</span>
<span class="p">{</span><span class="ss">controller: </span><span class="s1">'products'</span><span class="p">,</span> <span class="ss">action: </span><span class="s1">'edit'</span><span class="p">},</span>
<span class="p">{</span><span class="ss">controller: </span><span class="s1">'export'</span><span class="p">,</span> <span class="ss">action: </span><span class="s1">'products'</span><span class="p">},</span>
<span class="p">])</span>
<span class="c1"># 👍 /products products#index</span>
<span class="c1"># 👍 /products/1/edit products#edit</span>
<span class="c1"># 👍 /collections/1/products products#index</span>
<span class="c1"># 👍 /collections collections#index</span>
<span class="c1"># 👍 /export/products export#products</span>
<span class="c1"># ❌ /products/1 products#show</span>
<span class="c1"># ❌ /collections/1 collections#show</span>
</code></pre></div></div>
<hr />
<h3 id="spec">Spec</h3>
<p>I’ve also written a spec for this if you’d like to include it in your
application.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># spec/lib/active_link_path_spec.rb</span>
<span class="c1"># frozen_string_literal: true</span>
<span class="nb">require</span> <span class="s2">"spec_helper"</span>
<span class="nb">require</span> <span class="s2">"active_support/hash_with_indifferent_access"</span>
<span class="nb">require</span> <span class="s2">"./lib/active_link_path"</span>
<span class="n">describe</span> <span class="no">ActiveLinkPath</span> <span class="k">do</span>
<span class="n">context</span> <span class="s2">"active_match?"</span> <span class="k">do</span>
<span class="n">it</span> <span class="s2">"finds overlappings key value pairs"</span> <span class="k">do</span>
<span class="n">matcher</span> <span class="o">=</span> <span class="no">ActiveLinkPath</span><span class="p">.</span><span class="nf">new</span><span class="p">({</span><span class="s1">'key'</span> <span class="o">=></span> <span class="s1">'value1'</span><span class="p">,</span> <span class="s1">'key2'</span> <span class="o">=></span> <span class="s1">'value2'</span><span class="p">})</span>
<span class="n">expect</span><span class="p">(</span><span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">(</span><span class="ss">key: </span><span class="s1">'value1'</span><span class="p">)).</span><span class="nf">to</span> <span class="n">be_truthy</span>
<span class="n">expect</span><span class="p">(</span><span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">(</span><span class="ss">key: </span><span class="s1">'value2'</span><span class="p">)).</span><span class="nf">to</span> <span class="n">be_falsey</span>
<span class="k">end</span>
<span class="n">it</span> <span class="s2">"finds from multiple matching pairs"</span> <span class="k">do</span>
<span class="n">matcher</span> <span class="o">=</span> <span class="no">ActiveLinkPath</span><span class="p">.</span><span class="nf">new</span><span class="p">({</span><span class="s1">'key'</span> <span class="o">=></span> <span class="s1">'value1'</span><span class="p">,</span> <span class="s1">'key2'</span> <span class="o">=></span> <span class="s1">'value2'</span><span class="p">})</span>
<span class="n">expect</span><span class="p">(</span><span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">(</span><span class="ss">key: </span><span class="s1">'value1'</span><span class="p">,</span> <span class="ss">key2: </span><span class="s1">'value2'</span><span class="p">)).</span><span class="nf">to</span> <span class="n">be_truthy</span>
<span class="n">expect</span><span class="p">(</span><span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">(</span><span class="ss">key: </span><span class="s1">'value1'</span><span class="p">,</span> <span class="ss">key2: </span><span class="s1">'value1'</span><span class="p">)).</span><span class="nf">to</span> <span class="n">be_falsey</span>
<span class="n">expect</span><span class="p">(</span><span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">(</span><span class="ss">key: </span><span class="s1">'value1'</span><span class="p">,</span> <span class="ss">key3: </span><span class="s1">'value3'</span><span class="p">)).</span><span class="nf">to</span> <span class="n">be_falsey</span>
<span class="k">end</span>
<span class="n">it</span> <span class="s2">"allows an array of values"</span> <span class="k">do</span>
<span class="n">matcher</span> <span class="o">=</span> <span class="no">ActiveLinkPath</span><span class="p">.</span><span class="nf">new</span><span class="p">({</span><span class="s1">'key'</span> <span class="o">=></span> <span class="s1">'value1'</span><span class="p">,</span> <span class="s1">'key2'</span> <span class="o">=></span> <span class="s1">'value2'</span><span class="p">})</span>
<span class="n">expect</span><span class="p">(</span><span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">(</span><span class="ss">key: </span><span class="p">[</span><span class="s1">'value'</span><span class="p">,</span> <span class="s1">'value1'</span><span class="p">])).</span><span class="nf">to</span> <span class="n">be_truthy</span>
<span class="n">expect</span><span class="p">(</span><span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">(</span><span class="ss">key: </span><span class="p">[</span><span class="s1">'value'</span><span class="p">,</span> <span class="s1">'value2'</span><span class="p">])).</span><span class="nf">to</span> <span class="n">be_falsey</span>
<span class="k">end</span>
<span class="n">it</span> <span class="s2">"allows an array of matchers"</span> <span class="k">do</span>
<span class="n">matcher</span> <span class="o">=</span> <span class="no">ActiveLinkPath</span><span class="p">.</span><span class="nf">new</span><span class="p">({</span><span class="s1">'key'</span> <span class="o">=></span> <span class="s1">'value1'</span><span class="p">,</span> <span class="s1">'key2'</span> <span class="o">=></span> <span class="s1">'value2'</span><span class="p">})</span>
<span class="n">expect</span><span class="p">(</span>
<span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">([</span>
<span class="p">{</span><span class="ss">key: </span><span class="p">[</span><span class="s1">'value'</span><span class="p">],</span> <span class="ss">key2: </span><span class="s1">'value3'</span><span class="p">},</span>
<span class="p">{</span><span class="ss">key: </span><span class="s1">'value1'</span><span class="p">,</span> <span class="ss">key2: </span><span class="p">[</span><span class="s1">'value'</span><span class="p">,</span> <span class="s1">'value2'</span><span class="p">]}</span>
<span class="p">])</span>
<span class="p">).</span><span class="nf">to</span> <span class="n">be_truthy</span>
<span class="n">expect</span><span class="p">(</span>
<span class="n">matcher</span><span class="p">.</span><span class="nf">active_match?</span><span class="p">([</span>
<span class="p">{</span><span class="ss">key: </span><span class="p">[</span><span class="s1">'value'</span><span class="p">],</span> <span class="ss">key2: </span><span class="s1">'value3'</span><span class="p">},</span>
<span class="p">{</span><span class="ss">key: </span><span class="s1">'value1'</span><span class="p">,</span> <span class="ss">key2: </span><span class="p">[</span><span class="s1">'value'</span><span class="p">,</span> <span class="s1">'value1'</span><span class="p">]}</span>
<span class="p">])</span>
<span class="p">).</span><span class="nf">to</span> <span class="n">be_falsey</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>I’m often looking for a flexible solution for determining if I’m on the current route. I say flexible because sometimes a link is bound to a single controller, or sometimes multiple controllers, or multiple controllers with specific actions.Who Knows Who or What?2018-08-09T09:38:43+00:002018-08-09T09:38:43+00:00http://baylorrae.com/who-knows-who-or-what<p>A few years ago I had the opportunity to help build a social media platform
designed to connect professionals with various connections. The idea being to
index users based on companies or individuals they had previously mentioned
having a connection to.</p>
<p>While this is a problem I still don’t fully understand it was especially
difficult for me back then. I’m writing this post for my future self and for
others facing a similar issue.</p>
<h3 id="the-solution">The Solution</h3>
<p>I think the end goal to this problem is basically building a graph database that
has the following connections <code class="language-plaintext highlighter-rouge">user -> (person, company or location)</code>. With this
design it is possible to perform a reverse lookup to find the fellow user with
the contacts you are looking for.</p>
<h3 id="problem-1-classifying-texts">Problem #1: Classifying Texts</h3>
<p>The platform I worked on was strictly driven by unlabelled text. Posts could
be any of the following types <strong>Question</strong>, <strong>Announcement</strong>, <strong>Search</strong>,
<strong>Poll</strong> and didn’t follow a template.</p>
<blockquote>
<p><em>1.</em> “Does anyone have any recommendations for restaurants in X city?”</p>
</blockquote>
<blockquote>
<p><em>2.</em> “The company I work for is wanting to hire an intern for the summer. Please
contact me at “x@y.com”</p>
</blockquote>
<blockquote>
<p><em>3.</em> “I’m looking for a contact at X company to promote Y product.”</p>
</blockquote>
<blockquote>
<p><em>4.</em> “Which of these products do you think provide the best lift X, Y or Z?”</p>
</blockquote>
<p>All of these texts are hand written without any <em>required</em> structure. This
makes classifying and therefore indexing the important information very
difficult.</p>
<p>I think the way we solve this first problem is is by applying <a href="https://spacy.io/usage/linguistic-features#pos-tagging">Part of Speech</a>
tagging overlayed with an <a href="https://spacy.io/usage/training#intent-parser">intent parser</a>. The idea is we look at the verbs in
correlation to types of entities they are modifying/connected to. These two
techniques should help us label the text.</p>
<!-- Markup generated by spacy.displacy -->
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" id="0" class="displacy" viewBox="0 0 950 350" style="max-width: 100%; color: #000000; background: #ffffff; font-family: Arial; font-size: 30px">
<text class="displacy-token" fill="currentColor" text-anchor="middle" y="309.5">
<tspan class="displacy-word" fill="currentColor" x="50">find</tspan>
<tspan class="displacy-tag" dy="2em" fill="currentColor" x="50" style="font-size: 16px">ROOT</tspan>
</text>
<text class="displacy-token" fill="currentColor" text-anchor="middle" y="309.5">
<tspan class="displacy-word" fill="currentColor" x="225">a</tspan>
<tspan class="displacy-tag" dy="2em" fill="currentColor" x="225"></tspan>
</text>
<text class="displacy-token" fill="currentColor" text-anchor="middle" y="309.5">
<tspan class="displacy-word" fill="currentColor" x="400">hotel</tspan>
<tspan class="displacy-tag" dy="2em" fill="currentColor" x="400"></tspan>
</text>
<text class="displacy-token" fill="currentColor" text-anchor="middle" y="309.5">
<tspan class="displacy-word" fill="currentColor" x="575">with</tspan>
<tspan class="displacy-tag" dy="2em" fill="currentColor" x="575"></tspan>
</text>
<text class="displacy-token" fill="currentColor" text-anchor="middle" y="309.5">
<tspan class="displacy-word" fill="currentColor" x="750">good</tspan>
<tspan class="displacy-tag" dy="2em" fill="currentColor" x="750"></tspan>
</text>
<text class="displacy-token" fill="currentColor" text-anchor="middle" y="309.5">
<tspan class="displacy-word" fill="currentColor" x="925">wifi</tspan>
<tspan class="displacy-tag" dy="2em" fill="currentColor" x="925"></tspan>
</text>
<g class="displacy-arrow">
<path class="displacy-arc" id="arrow-0-1" stroke-width="2px" d="M70,264.5 C70,89.5 395.0,89.5 395.0,264.5" fill="none" stroke="currentColor" />
<text dy="1.25em" style="font-size: 16px; letter-spacing: 1px">
<textPath xlink:href="#arrow-0-1" class="displacy-label" startOffset="50%" fill="currentColor" text-anchor="middle">PLACE</textPath>
</text>
<path class="displacy-arrowhead" d="M395.0,266.5 L403.0,254.5 387.0,254.5" fill="currentColor" />
</g>
<g class="displacy-arrow">
<path class="displacy-arc" id="arrow-0-3" stroke-width="2px" d="M770,264.5 C770,177.0 915.0,177.0 915.0,264.5" fill="none" stroke="currentColor" />
<text dy="1.25em" style="font-size: 16px; letter-spacing: 2px">
<textPath xlink:href="#arrow-0-3" class="displacy-label" startOffset="50%" fill="currentColor" text-anchor="middle">QUALITY</textPath>
</text>
<path class="displacy-arrowhead" d="M770,266.5 L762,254.5 778,254.5" fill="currentColor" />
</g>
<g class="displacy-arrow">
<path class="displacy-arc" id="arrow-0-4" stroke-width="2px" d="M420,264.5 C420,2.0 925.0,2.0 925.0,264.5" fill="none" stroke="currentColor" />
<text dy="1.25em" style="font-size: 16px; letter-spacing: 1px">
<textPath xlink:href="#arrow-0-4" class="displacy-label" startOffset="50%" fill="currentColor" text-anchor="middle">ATTRIBUTE</textPath>
</text>
<path class="displacy-arrowhead" d="M925.0,266.5 L933.0,254.5 917.0,254.5" fill="currentColor" />
</g>
</svg>
<h3 id="problem-2-extracting-entities">Problem #2: Extracting Entities</h3>
<p><em>Once we have labelled the text, next we need to extract the entities in
question.</em></p>
<p>All of the above texts have important information to build our connections
database. In the first text, a question, we ask for recommendations in a city.
This is helps us identity that the user is likely making a trip to X city and
probably knows or would like to meet people in that city.</p>
<p>This information compounds itself since we will have likely extracted entities
from previous posts which would help us determine their likes and interests.</p>
<blockquote>
<p>“The following people are from X city and also like A, B and C.”</p>
</blockquote>
<p>This makes it realistic for us to create ‘answers’ or help other users find
questions they may be able to answer.</p>
<h3 id="example-entity-extraction">Example Entity Extraction</h3>
<!-- Markup generated by spacy.displacy -->
<div class="entities" style="line-height: 2.5; margin-bottom: 1.5em;
font-family: sans-serif; background-color: #f2f2f2; text-align: center; padding:
0.5em">
<mark class="entity" style="background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em; box-decoration-break: clone; -webkit-box-decoration-break: clone">
Apple
<span style="font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem">ORG</span>
</mark>
is looking at buying
<mark class="entity" style="background: #feca74; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em; box-decoration-break: clone; -webkit-box-decoration-break: clone">
U.K.
<span style="font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem">GPE</span>
</mark>
startup for
<mark class="entity" style="background: #e4e7d2; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em; box-decoration-break: clone; -webkit-box-decoration-break: clone">
$1 billion
<span style="font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem">MONEY</span>
</mark>
</div>
<h3 id="problem-3-finding-answers">Problem #3: Finding Answers</h3>
<p>The last step to finding answers comes from inferring connections. This is
probably best solved with word embeddings such as <a href="https://en.wikipedia.org/wiki/Word2vec">Word2Vec</a>.</p>
<p>I think there’s a good possibility we can start with a pretrained model from the
Wikipedia and/or Google News datasets. These models will help us gain insight
into the people, companies or locations and figure out how they are related.</p>
<p><img style="width: 100%" src="/assets/word2vec.png" alt="Word2Vec" /></p>
<p>The idea being that since we know the entities in a question (both the term and
the label) we can query our datasets to make predictions of who might have a
connection based on the relationships found in an unsupervised model from a
larger dataset.</p>
<p>It becomes even more interesting when you factor in that Word2Vec can also
perform mathematical operations to determine the resulting point. As we
improve our tagging of text it might be possible to create positive and negative
terms when searching for the most similar results.</p>
<p><img style="width: 100%" src="/assets/linear-relationships.png" alt="Word2Vec" /></p>
<h3 id="conclusion">Conclusion</h3>
<p>This post has been a bit abstract mostly because I haven’t actually implemented
any of the things I mentioned above. After spending the last few months working
on document classification and working with <a href="https://spacy.io">Spacy</a> I was reminded
of this previous project and thought I should log my initial ideas on solving
this problem again.</p>
<p>Sources:<br />
Model: https://codepen.io/explosion/pen/991f245ef90debb78c8fc369294f75ad<br />
Image: https://www.samyzaf.com/ML/nlp/word2vec2.png<br />
Image: https://www.tensorflow.org/tutorials/representation/word2vec</p>A few years ago I had the opportunity to help build a social media platform designed to connect professionals with various connections. The idea being to index users based on companies or individuals they had previously mentioned having a connection to.Embeddings in Machine Learning2018-07-31T02:29:43+00:002018-07-31T02:29:43+00:00http://baylorrae.com/embeddings-in-machine-learning<p>I’ve recently been tasked with applying machine learning and neural networks to
classify documents at my day job. Everyday while working to solve this problem I
have new realizations about this field which was completely foreign to me only a
few months ago. I took what is called a top-down approach, which basically means
diving in head first and trying to solve problems without necessarily
understanding the ins and outs.</p>
<p>Interestingly enough this isn’t all that different than how I learned web
development. I started by viewing the source of a web page, then creating my own
<code class="language-plaintext highlighter-rouge">.html</code> file, followed by applying CSS and JavaScript to build what I wanted.</p>
<p>However, neural networks and machine learning require far more discipline than
web development. While I would argue the tools, frameworks and blogs are equally
available, the actual pieces are often just assumed or are given without any
explanation or application.</p>
<h3 id="what-are-embedding-layers">What are Embedding Layers</h3>
<p>One of these key components are Embedding Layers. Rutger Ruizendaal wrote <a href="https://towardsdatascience.com/deep-learning-4-embedding-layers-f9a02d55ac12">a
fantastic article</a> explaining Embedding layers
in far more detail than I will here. My goal is to hopefully help others who
struggle on how to make them applicable.</p>
<p>An embedding can be thought of a way to provide dimension to anything. Commonly
it is applied with word embeddings such as <a href="https://nlp.stanford.edu/projects/glove/">GloVe</a> and <a href="https://en.wikipedia.org/wiki/Word2vec">Word2Vec</a>. I found this
difficult to grasp as those two projects provide dictionaries with 100, 200 and
300 dimensions.</p>
<p>I read and understood that those large dimensions were designed to provide
context to each word in the vocabulary. This is done to provide similarities and
distinctions to better understand the word within the context of a sentence or
phrase. For instance, the company “Apple” and fruit “apple” are the same word
but depending on the context are completely different entities.</p>
<h3 id="how-to-apply-embedding-layers">How to Apply Embedding Layers</h3>
<p><img style="float: right; margin-left: 15px; width: 148px;" src="/assets/tomato.jpg" alt="Tomato" /></p>
<p>When discussed you’ll often read that embeddings can be applied to almost
everything. But what? It makes sense with vocabulary but can they also be
applied to something such as food or an ingredient? Yes!</p>
<blockquote>
<p>An embedding can be thought of as a list of floats that align to a set of
features</p>
</blockquote>
<p>Let’s say we wanted to classify a recipe as <code class="language-plaintext highlighter-rouge">Breakfast</code>, <code class="language-plaintext highlighter-rouge">Lunch</code>,
<code class="language-plaintext highlighter-rouge">Dinner</code> or <code class="language-plaintext highlighter-rouge">Desert</code>. One way we might solve this is by using embeddings on each
of the ingredients of a recipe. This would allow us to extrapolate the type of
recipe beyond looking at the frequency of shared ingredients.</p>
<p>We could look at an ingredient by classifying features such as <code class="language-plaintext highlighter-rouge">Citrus</code>,
<code class="language-plaintext highlighter-rouge">Sweet</code>, <code class="language-plaintext highlighter-rouge">Sour</code>, <code class="language-plaintext highlighter-rouge">Tangy</code>, <code class="language-plaintext highlighter-rouge">Solidity</code>, and <code class="language-plaintext highlighter-rouge">Water Saturation</code>. Most ingredients
are not just one of these things, rather, they are a combination of all of them.</p>
<p><img style="width: 250px; float: left; margin-right: 15px" src="/assets/tomato-radar-chart.png" alt="Tomato Radar Chart" /></p>
<p>The radar chart to the left visualizes how I might embed the details of a tomato
with the above features.</p>
<div style="clear: left"></div>
<p>In code you would represent this as the following. The
values themselves are relative to the feature they represent.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">ingredients</span> <span class="o">=</span> <span class="p">{</span>
<span class="s">'tomato'</span><span class="p">:</span> <span class="p">[</span>
<span class="mf">0.35</span><span class="p">,</span> <span class="c1"># Citrus
</span> <span class="mf">0.54</span><span class="p">,</span> <span class="c1"># Sweet
</span> <span class="mf">0.44</span><span class="p">,</span> <span class="c1"># Sour
</span> <span class="mf">0.64</span><span class="p">,</span> <span class="c1"># Tangy
</span> <span class="mf">0.62</span><span class="p">,</span> <span class="c1"># Solidity
</span> <span class="mf">0.73</span> <span class="c1"># Water Saturation
</span> <span class="p">],</span>
<span class="s">'broccoli'</span><span class="p">:</span> <span class="p">[</span>
<span class="mf">0.02</span><span class="p">,</span> <span class="c1"># Citrus
</span> <span class="mf">0.09</span><span class="p">,</span> <span class="c1"># Sweet
</span> <span class="mf">0.13</span><span class="p">,</span> <span class="c1"># Sour
</span> <span class="mf">0.14</span><span class="p">,</span> <span class="c1"># Tangy
</span> <span class="mf">0.87</span><span class="p">,</span> <span class="c1"># Solidity
</span> <span class="mf">0.32</span> <span class="c1"># Water Saturation
</span> <span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="why-is-this-necessary">Why is this necessary</h3>
<p>After giving this some thought I realized this is necessary since the neural
network we are training (in my case at least) has no prior knowledge. It can
only learn from the data we provide. This is a polar opposite to us. We have the
ability to create ‘embeddings’ on everything we come into contact with. Whether
it’s food, cars, software, television shows or even other people.</p>
<p>This is an important factor that I think can be easily overlooked. By
understanding that everything is built up by many, small features we can learn
how to represent our data to assist with training a successful neural network.</p>
<h3 id="off-topic">Off topic…</h3>
<p>In my opinion as of 7/30/2018, the step to successful machine learning is trying
to replicate the our own natural ability to collect information and make
informed decisions. Neural networks never have 100% accuracy since we are
modeling them after ourselves and we can expect nothing more considering our
own limitations. I find this to be an interesting dynamic that generally isn’t
shared with software development.</p>I’ve recently been tasked with applying machine learning and neural networks to classify documents at my day job. Everyday while working to solve this problem I have new realizations about this field which was completely foreign to me only a few months ago. I took what is called a top-down approach, which basically means diving in head first and trying to solve problems without necessarily understanding the ins and outs.Testing APIs2017-01-25T03:25:00+00:002017-01-25T03:25:00+00:00http://baylorrae.com/testing-apis<p>I recently had the opportunity to test an API and wanted to validate the
results along with the structure of the response. When looking around on the
internet for inspiration on how to solve this I was intrigued by the wonderful
DSL created by <a href="https://github.com/vlucas/frisby#creating-tests">Frisby</a>.</p>
<p>My goal was to not only test the security and correctness internally but also
the “user interface” of response codes and structure. I think the later is
extremely important due to the nature of objects being easily changeable
without any restrictions by the framework.</p>
<p>Frisby’s DSL solves this by having chainable method calls to validate the <code class="language-plaintext highlighter-rouge">http
status</code>, <code class="language-plaintext highlighter-rouge">json schema</code> and <code class="language-plaintext highlighter-rouge">json body</code>. All three of which are the “user
interface” our customers would be looking at on a daily basis.</p>
<p>The following class provides a similar DSL for validating requests sent with <code class="language-plaintext highlighter-rouge">Rack::Test</code>. The JSON schema is validated with the <a href="https://github.com/ruby-json-schema/json-schema">json-schema</a> gem.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">ValidateResponse</span>
<span class="kp">include</span> <span class="no">RSpec</span><span class="o">::</span><span class="no">Matchers</span>
<span class="nb">attr_reader</span> <span class="ss">:response</span>
<span class="n">delegate</span> <span class="ss">:status_code</span><span class="p">,</span> <span class="ss">to: </span><span class="no">Rack</span><span class="o">::</span><span class="no">Utils</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">response</span><span class="p">)</span>
<span class="vi">@response</span> <span class="o">=</span> <span class="n">response</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">with</span><span class="p">(</span><span class="n">response</span><span class="p">)</span>
<span class="n">new</span><span class="p">(</span><span class="n">response</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">expect_status</span><span class="p">(</span><span class="n">status</span><span class="p">)</span>
<span class="n">expect</span><span class="p">(</span><span class="n">status_code</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="nf">status</span><span class="p">)).</span><span class="nf">to</span> <span class="n">eq</span><span class="p">(</span><span class="n">status_code</span><span class="p">(</span><span class="n">status</span><span class="p">))</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">expect_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">)</span>
<span class="n">errors</span> <span class="o">=</span> <span class="no">JSON</span><span class="o">::</span><span class="no">Validator</span><span class="p">.</span><span class="nf">fully_validate</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">response</span><span class="p">.</span><span class="nf">body</span><span class="p">)</span>
<span class="k">if</span> <span class="n">errors</span><span class="p">.</span><span class="nf">any?</span>
<span class="k">raise</span> <span class="no">RSpec</span><span class="o">::</span><span class="no">Expectations</span><span class="o">::</span><span class="no">ExpectationNotMetError</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">errors</span><span class="p">)</span>
<span class="k">end</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">expect_body</span><span class="p">(</span><span class="n">body</span><span class="p">)</span>
<span class="n">expect</span><span class="p">(</span><span class="no">JSON</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="nf">body</span><span class="p">)).</span><span class="nf">to</span> <span class="n">eq</span><span class="p">(</span><span class="n">body</span><span class="p">)</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Using this class is very simple and combined with the <a href="http://www.rubydoc.info/github/brynary/rack-test/Rack/MockSession:last_response"><code class="language-plaintext highlighter-rouge">last_response</code></a> method provided by <code class="language-plaintext highlighter-rouge">Rack::Test</code> makes this a breeze.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">ValidateResponse</span><span class="p">.</span><span class="nf">with</span><span class="p">(</span><span class="n">last_response</span><span class="p">)</span>
<span class="p">.</span><span class="nf">expect_status</span><span class="p">(</span><span class="ss">:ok</span><span class="p">)</span>
<span class="p">.</span><span class="nf">expect_schema</span><span class="p">(</span><span class="s2">"./features/api/schemas/projects/index.json"</span><span class="p">)</span>
<span class="p">.</span><span class="nf">expect_body</span><span class="p">([</span>
<span class="p">{</span>
<span class="s2">"id"</span> <span class="o">=></span> <span class="mi">1</span><span class="p">,</span>
<span class="s2">"title"</span> <span class="o">=></span> <span class="s2">"project-title"</span><span class="p">,</span>
<span class="s2">"created_at"</span> <span class="o">=></span> <span class="s2">"2017-01-01T00:00:00.000Z"</span><span class="p">,</span>
<span class="s2">"updated_at"</span> <span class="o">=></span> <span class="s2">"2017-01-01T00:00:00.000Z"</span>
<span class="p">}</span>
<span class="p">])</span>
</code></pre></div></div>
<p>With the <code class="language-plaintext highlighter-rouge">ValidateResponse</code> class in place validating the important public facing parts of our API can be done quickly with a pleasant DSL.</p>I recently had the opportunity to test an API and wanted to validate the results along with the structure of the response. When looking around on the internet for inspiration on how to solve this I was intrigued by the wonderful DSL created by Frisby.Adding a User Manager to your Feature Tests2016-11-18T14:30:14+00:002016-11-18T14:30:14+00:00http://baylorrae.com/user-manager<p>In my last post I described how to <a href="/writing-acceptance-tests-in-third-person">write declarative step definitions</a>.
However, this means our test cases can’t assume it knows who is performing the
action because the user can change at any time. We’ll need to introduce a sort
of state machine to track the current user and log in as another user when
needed.</p>
<p>I’ve written this example feature to purchase a product using a <strong>buyer</strong> and
<strong>seller</strong> with the <strong>buyer</strong> performing two steps in a row.</p>
<div class="language-gherkin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">Feature</span><span class="p">:</span> purchase product
<span class="kn">Scenario</span><span class="p">:</span> purchase product
<span class="nf">Given </span>seller has created a product
<span class="nf">When </span>buyer purchases the product
<span class="nf">And </span>buyer sends payment
<span class="nf">Then </span>seller should receive money
<span class="nf">And </span>the product shouldn't be listed
</code></pre></div></div>
<p>When writing our step definitions we will include a capture group to get the
username and then pass that to our <code class="language-plaintext highlighter-rouge">UserManager</code> class.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Given</span><span class="p">(</span><span class="sr">/^(.*) has created a product$/</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">username</span><span class="o">|</span>
<span class="no">UserManager</span><span class="p">.</span><span class="nf">change_user</span><span class="p">(</span><span class="n">username</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">When</span><span class="p">(</span><span class="sr">/^(.*) purchases the product$/</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">username</span><span class="o">|</span>
<span class="no">UserManager</span><span class="p">.</span><span class="nf">change_user</span><span class="p">(</span><span class="n">username</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">When</span><span class="p">(</span><span class="sr">/^(.*) sends payment$/</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">username</span><span class="o">|</span>
<span class="no">UserManager</span><span class="p">.</span><span class="nf">change_user</span><span class="p">(</span><span class="n">username</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">Then</span><span class="p">(</span><span class="sr">/^(.*) should receive money$/</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">username</span><span class="o">|</span>
<span class="no">UserManager</span><span class="p">.</span><span class="nf">change_user</span><span class="p">(</span><span class="n">username</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">Then</span><span class="p">(</span><span class="sr">/^the product shouldn't be listed$/</span><span class="p">)</span> <span class="k">do</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now for the final piece, the <code class="language-plaintext highlighter-rouge">UserManager</code> class. This is a very simple class
with a single method to update the <code class="language-plaintext highlighter-rouge">current_username</code> variable if it is
different. When this variable changes it will tell our <code class="language-plaintext highlighter-rouge">LoginPage</code> to log us in
with the new username.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">LoginPage</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">login_as</span><span class="p">(</span><span class="n">username</span><span class="p">)</span>
<span class="c1"># fill in login fields and submit form here</span>
<span class="nb">puts</span> <span class="s2">"Logged in as </span><span class="si">#{</span><span class="n">username</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">UserManager</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">change_user</span><span class="p">(</span><span class="n">username</span><span class="p">)</span>
<span class="k">return</span> <span class="k">if</span> <span class="vi">@current_username</span> <span class="o">==</span> <span class="n">username</span>
<span class="vi">@current_username</span> <span class="o">=</span> <span class="n">username</span>
<span class="no">LoginPage</span><span class="p">.</span><span class="nf">login_as</span><span class="p">(</span><span class="n">username</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>As you can see this is a very small change to make and allows our step
definitions to simply carry out the instructions from the high level scenario.</p>In my last post I described how to write declarative step definitions. However, this means our test cases can’t assume it knows who is performing the action because the user can change at any time. We’ll need to introduce a sort of state machine to track the current user and log in as another user when needed.