Hi everyone, this is my first post on CodePen! I thought I'd share my collection of demos with my new library: SeeClarke.js

SeeClarke.js is a library that positions a cursor on the screen via head tracking, and is powered by PoseNet via Tensorflow.js - a hugely popular machine learning library by Google. I began working on this project while at a homeless shelter in order to help a friend who was cut off from communicating with his family following a stroke. You can view my projects story in this thread on Twitter.

The library abstracts away all the machine learning stuff, leaving you with just the cursor positions. With this, you can do a lot (basically anything you can do on a mobile device with one finger can be done via head tracking). Here are a few of my first demos:

EDIT: OH! These are best viewed full screen, the reason is that I didn't plan for SeeClarke to run in iFrames until I just reread this post. This will be automatically fixed in the next release!

Look around YouTube 360 videos:

Look around Street Views:

Pan around Google Maps:

Interact with HTML elements via native events:

To use SeeClarke:

  • Simply add the library to your pen: <script src="https://unpkg.com/seeclarke/dist/seeclarke.js"></script>
  • Initialize the library, const seeclarke = new SeeClarke(OPTIONS);
  • Listen to the onSeeClarkePoseUpdates and run your code:
  // Step 3: Position the cursor
window.addEventListener('onSeeClarkePoseUpdates', (ev) => {
  // Grab the SeeClarke instance that called this
  const context = ev.detail.context

  // You can now access any of the properties, including the poses
  const poses = context.poses

  // Loop through poses to get their details
  poses.forEach((pose, index) => {
    let $el
    let x = pose.pointedAt.x
    let y = pose.pointedAt.y
    // ... do stuff ...

See this pen for the most basic implementation with a cursor (you don't need to show a cursor):

Let me know what you think, my goal is to make 100 demos! Oz @labofoz

2,226 2 16