cssAudio - Activefile-genericCSS - ActiveGeneric - ActiveHTML - ActiveImage - ActiveJS - ActiveSVG - ActiveText - Activefile-genericVideo - ActiveLovehtmlicon-new-collectionicon-personicon-teamlog-outoctocatpop-outspinnerstartv

Video Music

A Pen By Jake Albaugh Pro

Pen Settings

About HTML Preprocessors

HTML preprocessors can make writing HTML more powerful or convenient. For instance, Markdown is designed to be easier to write and read for text documents and you could write a loop in Pug.

Learn more · Versions

Adding Classes

In CodePen, whatever you write in the HTML editor is what goes within the <body> tags in a basic HTML5 template. So you don't have access to higher-up elements like the <html> tag. If you want to add classes there that can affect the whole document, this is the place to do it.

About the <head>

In CodePen, whatever you write in the HTML editor is what goes within the <body> tags in a basic HTML5 template. If you need things in the <head> of the document, put that code here.

!
Insecure Resource

The resource you are linking to is using the 'http' protocol, which may not work when the browser is using https.

About CSS Preprocessors

CSS preprocessors help make authoring CSS easier. All of them offer things like variables and mixins to provide convenient abstractions.

Learn more · Versions

CSS Base

About CSS Base

It's a common practice to apply CSS to a page that styles elements such that they are consistent across all browsers. We offer two of the most popular choices: normalize.css and a reset. Or, choose Neither and nothing will be applied.

     

Vendor Prefixing

About Vendor Prefixing

To get the best cross-browser support, it is a common practice to apply vendor prefixes to CSS properties and values that require them to work. For instance -webkit- or -moz-.

We offer two popular choices: Autoprefixer (which processes your CSS server-side) and -prefix-free (which applies prefixes via a script, client-side).

     

Add External Stylesheets/Pens

Any URL's added here will be added as <link>s in order, and before the CSS in the editor. If you link to another Pen, it will include the CSS from that Pen. If the preprocessor matches, it will attempt to combine them before processing.

Powered by Algolia
About External Resources

You can apply CSS to your Pen from any stylesheet on the web. Just put a URL to it here and we'll apply it, in the order you have them, before the CSS in the Pen itself.

If the stylesheet you link to has the file extension of a preprocessor, we'll attempt to process it before applying.

You can also link to another Pen here, and we'll pull the CSS from that Pen and include it. If it's using a matching preprocessor, we'll combine the code before preprocessing, so you can use the linked Pen as a true dependency.

Learn more

+ add another resource

About JavaScript Preprocessors

JavaScript preprocessors can help make authoring JavaScript easier and more convenient. For instance, CoffeeScript can help prevent easy-to-make mistakes and offer a cleaner syntax and Babel can bring ECMAScript 6 features to browsers that only support ECMAScript 5.

Learn more · Versions

You're using npm packages, so we've auto-selected Babel for you here, which we require to process imports and make it all work. If you need to use a different JavaScript preprocessor, remove the packages in the npm tab.

Add External Scripts/Pens

Any URL's added here will be added as <script>s in order, and run before the JavaScript in the editor. You can use the URL of any other Pen and it will include the JavaScript from that Pen.

Powered by Algolia
About External Resources

You can apply a script from anywhere on the web to your Pen. Just put a URL to it here and we'll add it, in the order you have them, before the JavaScript in the Pen itself.

If the script you link to has the file extension of a preprocessor, we'll attempt to process it before applying.

You can also link to another Pen here, and we'll pull the JavaScript from that Pen and include it. If it's using a matching preprocessor, we'll combine the code before preprocessing, so you can use the linked Pen as a true dependency.

Learn more

+ add another resource

Use npm Packages

We can make npm packages available for you to use in your JavaScript. We use webpack to prepare them and make them available to import. We'll also process your JavaScript with Babel.

⚠️ This feature can only be used by logged in users.

Code Indentation

     

Save Automatically?

If active, Pens will autosave every 30 seconds after being saved once.

Auto-Updating Preview

If enabled, the preview panel updates automatically as you code. If disabled, use the "Run" button to update.

HTML

            
              <header></header>

<section>
  <div><video src="" crossorigin="anonymous" autoplay></video></div>
  <canvas></canvas>
</section>

<footer>
  <p>
    This makes music and needs access to your camera.
    The average hue from your camera controls which chord will play.
  </p>
  <p><button id="button"><span>Start</span><span>Stop</span></button></p>
</footer>

<div class="trydebug hide">
  Bummer, you cant load this here. You can open
  <a href="https://codepen.io/jakealbaugh/debug/ZxLKvG" target="blank">
    this page
  </a> in a new window which might fix the problem.
</div>
!

CSS

            
              html, body {
  height: 100%;
  overflow: hidden;
}

body, section {
  background: #111;
}

$size: 300px;

div.trydebug {
  position: absolute;
  top: 50%;
  left: 50%;
  box-sizing: border-box;
  z-index: 10;
  padding: 4rem;
  width: 400px;
  max-width: 95%;
  transform: translate(-50%, -50%);
  background: black;
  color: white;
  text-align: center;
  border: 2px solid tomato;
  a {
    color: tomato;
  }
  &.hide {
    display: none;
  }
}

section {
  height: $size * 2;
  width: $size;
  position: absolute;
  left: calc(50% - #{$size * 0.5});
  top: calc(50% - #{$size});
  flex-direction: column;
  display: flex;
  box-shadow: 0px 4px 24px 4px rgba(0, 0, 0, 0.1);
  border-radius: 4px;
  overflow: hidden;
  div, canvas {
    height: $size;
    width: $size;
    display: block;
  }
  div {
    overflow: hidden;
    position: relative;
    video {
      display: block;
      position: absolute;
      top: 0;
      left: 50%;
      transform: translateX(-50%);
      width: auto;
      height: 100%;
    }
  }

  canvas {
    image-rendering: pixelated;
  }
}


header {
  position: fixed;
  font-family: system, Helvetica, Arial, sans-serif;
  color: white;
  width: $size;
  left: calc(50% - #{$size * 0.5});
  top: calc(50% + #{$size + 10});
  display: flex;
  > div {
    flex: 1;
    height: 20px;
    position: relative;
    display: flex;
    border-radius: 4px;
    box-shadow: 0px 4px 24px 4px rgba(0, 0, 0, 0.1);
    overflow: hidden;
    > span {
      flex: 1;
    }
    > span:last-child {
      position: absolute;
      display: block;
      top: 0;
      width: 2px;
      height: 20px;
      background: white;
      transform: translateX(-50%);
    }
  }
}

footer {
  position: absolute;
  bottom: 0;
  left: 50%;
  box-sizing: border-box;
  width: 100%;
  padding: 12px;
  text-align: center;
  transform: translateX(-50%);
  max-width: $size * 2;
  color: white;
  text-shadow: 2px 2px 0px rgba(0,0,0,0.2);
  -webkit-font-smoothing: antialiased;
	-moz-osx-font-smoothing: grayscale;
  button {
    appearance: none;
    border: none;
    padding: 12px 24px;
    border-radius: 4px;
    box-shadow: 0px 2px 0px 1px rgba(0, 0, 0, 0.3);
    background: black;
    color: white;
    cursor: pointer;
    font-size: 24px;
    // text-transform: uppercase;
    font-weight: bold;
    &.active {
      background: white;
      color: black;
    }
    &:active {
      box-shadow: 0px 0px 0px 1px rgba(0, 0, 0, 0.3);
      transform: translateY(2px);
    }
    &:not(.active) span:last-child,
    &.active span:first-child {
      display: none;
    }
  }
}

@media (min-width: #{$size * 2}) {
  section {
    width: $size * 2;
    height: $size;
    flex-direction: row;
    left: calc(50% - #{$size});
    top: calc(50% - #{$size * 0.5});
  }
  header {
    top: calc(50% + #{$size * 0.5 + 10});
    width: $size * 2;
    left: calc(50% - #{$size});
  }
}
!

JS

            
              // console.clear();

class ColorMapper {
  constructor(di, vid, cvs) {
    this.di = di;
    this.cvs = cvs;
    this.ctx = cvs.getContext('2d');
    this.vid = vid;
    this.rgb = new Array(di * di);
    this.report = document.querySelector('header');
    this.avgRGB = null;
    this.avgHSL = null;
    this._initialize();
  }

  start(stream) {
    try {
      this.vid.srcObject = stream;
    } catch (error) {
      console.log(error)
      this.vid.src = URL.createObjectURL(stream);
    }
    this.vid.play();
    let self = this;
    this.vid.onloadedmetadata = function() { 
      self.w = self.vid.videoWidth / self.vid.videoHeight * self.cvs.width;
      self.vid.width = self.vid.videoWidth;
      self.vid.height = self.vid.videoHeight;
      self.offX = (self.w - self.cvs.width) * -0.5;
      self._animate();
    }
  }

  _animate() {
    window.requestAnimationFrame(this._animate.bind(this));
    this.ctx.drawImage(this.vid, this.offX, 0, this.w, this.di);
    this._processData();
  }
  
  _initialize() {
    this.cvs.height = this.di;
    this.cvs.width  = this.di;
  }
  
  _processData() {
    let imgData = this.ctx.getImageData(0, 0, this.di, this.di),
        data = imgData.data,
        len = data.length,
        avg = [0, 0, 0];
    for (let i = 0; i < len; i += 4) {
      let colorIdx = i * 0.25,
        newRgb = [data[i], data[i + 1], data[i + 2]],
        prevRgb = this.rgb[colorIdx],
        easeRgb = this._easeVals(prevRgb, newRgb);
      imgData.data[i + 0] = easeRgb[0];
      imgData.data[i + 1] = easeRgb[1];
      imgData.data[i + 2] = easeRgb[2];
      avg[0] += easeRgb[0];
      avg[1] += easeRgb[1];
      avg[2] += easeRgb[2];
      this.rgb[colorIdx] = easeRgb;
    }
    let colors = len * 0.25;
    avg[0] /= colors;
    avg[1] /= colors;
    avg[2] /= colors;
    this.avgRGB = avg;
    this.avgHSL = this._rgbToHsl(avg);
    document.body.style.backgroundColor = `rgb(${avg.map(Math.round).join(',')})`;
    this.ctx.putImageData(imgData, 0, 0);
    this.report.innerHTML = `
      ${this._gradientSpan('h', this.avgHSL[0])}
    `
  }
  
  _gradientSpan(type, value) {
    let gradient = [];
    
    if (type === 'h')
      for (let i = 0; i < 7; i++)
        gradient.push(`hsl(${i / 7 * 360}, 100%, 60%)`);
    else if (type === 's')
      for (let i = 0; i < 7; i++)
        gradient.push(`hsl(0, ${i / 7 * 100}%, 60%)`);
    else if (type === 'l')
      for (let i = 0; i < 7; i++)
        gradient.push(`hsl(0, 0%, ${i / 7 * 100}%)`);
    return `
      <div>
        ${gradient.map(g => `<span style="background-color: ${g}"></span>` ).join('')}
        <span style="left: ${value * 100}%"></span>
      </div>
    `;
  }
  
  _easeVals(prev, next, ease = 0.025) {
    if (!prev) return next;
    return next.map((n, i) => (n - prev[i]) * ease + prev[i]);
  }

  // https://stackoverflow.com/questions/2353211/hsl-to-rgb-color-conversion
  _rgbToHsl(rgb) {
    let [r, g, b] = rgb;
    r /= 255, g /= 255, b /= 255;
    let max = Math.max(r, g, b), min = Math.min(r, g, b);
    let h, s, l = (max + min) / 2;

    if (max == min) h = s = 0;
    else {
      let d = max - min;
      s = l > 0.5 ? d / (2 - max - min) : d / (max + min);
      switch (max) {
        case r: h = (g - b) / d + (g < b ? 6 : 0); break;
        case g: h = (b - r) / d + 2; break;
        case b: h = (r - g) / d + 4; break;
      }
      h /= 6;
    }

    return [h, s, l];
  }
}

class Score {
  constructor(map) {
    this.map = map;
    this.noteIdx = 0;
    this.tick = 0;
    this._initialize();
  }
  
  toggle() {
    if (this.on) {
      this.on = false;
      Tone.Transport.stop();
      this.tick = 0;
      this.synth.triggerRelease();
      this.mid.triggerRelease();
      this.bass1.triggerRelease();
      this.bass2.triggerRelease();
      document.querySelector('button').classList.remove('active');
    } else {
      this.on = true;
      Tone.Transport.start();
      document.querySelector('button').classList.add('active');
    }
  }

  step(time) {
    let hsl = this.map.avgHSL,
      safe = i => Math.max(0, i - 0.000000001);
    if (!hsl || !this.on) return;
    if (this.tick % 4 === 0) {
      this.noteIdx = Math.floor(safe(hsl[0]) * 7);
      this.hsl = hsl;
    }
    let note = this.note,
      pattern = [2, 0, 1, 0, 2, 1, 2, 0],
      octaves = [5, 5, 4, 4, 4, 5, 4, 4],
      arpNote = note[pattern[this.tick % 8]],
      arpOct = octaves[this.tick % 8];
    arpOct += arpNote[1];
    let arp = `${arpNote[0]}${arpOct}`;
    let mtd = (this.tick === 0) ? 'triggerAttack' : 'setNote';
    this.synth[mtd](arp, time);
    if (this.tick % 4 === 0) {
      let bass1 = note[0],
        bass2 = note[2],
        mid = note[1];
      this.mid[mtd](mid[0]+(mid[1]+3), time);
      this.bass1[mtd](bass1[0]+(bass1[1]+1), time);
      this.bass2[mtd](bass2[0]+(bass2[1]+1), time);
    }
    this.tick++;
  }

  get note() {
    return this.notes[this.noteIdx];
  }
  
  get notes() {
    return [
      [['G',  0], ['A#', 0], ['C#', 1]],
      [['A#', 0], ['C#', 1], ['F',  1]],
      [['C#', 1], ['F',  1], ['G#', 1]],
      [['F',  1], ['G#', 1], ['C',  2]],
      [['D#', 1], ['G',  1], ['A#', 1]],
      [['C',  1], ['D#', 1], ['G',  1]],
      [['G#', 0], ['C',  1], ['D#', 1]],
    ];
    // return [
    //   [['G',  0], ['A#', 0], ['C#', 1]],
    //   [['G#', 0], ['C',  1], ['D#', 1]],
    //   [['A#', 0], ['C#', 1], ['F',  1]],
    //   [['C',  1], ['D#', 1], ['G',  1]],
    //   [['C#', 1], ['F',  1], ['G#', 1]],
    //   [['D#', 1], ['G',  1], ['A#', 1]],
    //   [['F',  1], ['G#', 1], ['C',  2]],
    // ];
  }
  
  _initialize() {
    this.synth = new Tone.Synth({
      portamento: 0.00625,
      oscillator: { type: 'sine' },
      envelope: { release: 0.07 }
    });
    this.mid = new Tone.Synth({
      portamento: 0.00625,
      oscillator: { type: 'triangle' },
      envelope: { release: 0.07 }
    });
    this.bass1 = new Tone.FMSynth({
      portamento: 0.0125
    });
    this.bass2 = new Tone.FMSynth({
      portamento: 0.0125
    });
    let gain1 = new Tone.Gain(0.4),
      gain2 = new Tone.Gain(0.5),
      pan1 = new Tone.Panner(-1),
      pan2 = new Tone.Panner(1);
    this.synth.connect(gain1);
    this.mid.connect(gain1);
    pan1.connect(gain2);
    pan2.connect(gain2);
    this.bass1.connect(pan1);
    this.bass2.connect(pan2);
    gain1.toMaster();
    gain2.toMaster();
    Tone.Transport.scheduleRepeat(time => {
      this.step(time);
    }, '16n');
    Tone.Transport.bpm.value = 120;
  }
}

const vid = document.querySelector('video'),
  cvs = document.querySelector('canvas'),
  map = new ColorMapper(4, vid, cvs),
  score = new Score(map);

navigator.mediaDevices.getUserMedia({ audio: false, video: true })
  .then(stream => {
    map.start(stream);
    document.querySelector('button').addEventListener('click', () => score.toggle());
  })
  .catch(err => {
    if (window.location.href.match('debug')) {
      alert('Sorry. This wont work on your browser or device. Update or use Chrome/Firefox.')
    } else {
      document.querySelector('div.trydebug').classList.remove('hide');
    }
  });


StartAudioContext(Tone.context, '#button')
!
999px
🕑 One or more of the npm packages you are using needs to be built. You're the first person to ever need it! We're building it right now and your preview will start updating again when it's ready.
Loading ..................

Console