Trans Scend Survival

Trans: Latin prefix implying "across" or "Beyond", often used in gender nonconforming situations – Scend: Archaic word describing a strong "surge" or "wave", originating with 15th century english sailors – Survival: 15th century english compound word describing an existence only worth transcending.

Author: Jess (Page 2 of 4)

Warbler Trillers of the Charles

April 18, 2019 / / 0 Comments

The first ones to arrive in MA, brush up!

 

Palm warbler

songs

 

This is usually the first one to arrive.  Gold bird, medium sized warbler, rufus hat.  When they arrive in MA they are often found lower than usual / on the ground looking for anything they can munch on.  Song is a rapid trill. More “musical / pleasant” than a fast chipping sparrow, faster than many Junco trills.

 

Pine warbler

songs

 

Slimmer than palm, no hat, very slim beak, has streaks on the breast usually.  Also a triller. They remain higher in the trees on arrival.

 

Yellow-rumped warbler

songs

 

Spectacular bird, if it has arrived you can’t miss it- also they will arrive by the dozen so worth waiting for a good visual.  These also trill, which is another reason it is good to get a visual. The trill is slow, very “sing-song”, and has a downward inflection at the end.  If there are a bunch sticking around for the summer, try to watch some sing- soon enough you will be able to pick out this trill from the others.

 

-- Yellow warbler says “sweet sweet sweet, I’m so Sweet!” and can get a bit confusing with Yellow-rumped warbler

-- Chestnut-sided warbler says “very very pleased to meet ya!” and can get a bit confusing with Yellow warbler

 

Black-and-white warbler

songs

 

Looks like a zebra - always acts like a nuthatch (clings to trunk and branches).  This one trills like a rusty wheel. It can easily be distinguished after a bit of birding with some around.  

 

American Redstart

songs

 

Adult males look like a late 50’s hot-rodded American muscle car: long, low, two tone paint job.  Matte/luster black with flame accents. Can’t miss it. The females and young males are buff (chrome, to keep in style I guess) with yellow accents.  Look for behavior- if a “female” is getting beaten up while trying to sing a song in the same area, it is actually a first year male failing to establish a territory due to obviously being a youth.

 

Cheers,

- Jess

Notes on a Free and Open Source Notes App:  Joplin

March 21, 2019 / / 0 Comments

Joplin for all your Operating Systems and devices

As a lifelong IOS + OSX user (Apple products), I have used many, many notes apps over the years.  From big name apps like OmniFocus, Things 3, Notes+, to all the usual suspects like Trello, Notability, Notemaster, RTM, and others, I always eventually migrate back to Apple notes, simply because it is always available and always up to date.  There are zero “features” besides this convenience, which is why I am perpetually willing to give a new app a spin.

Joplin is free, open source, and works on OSX, Windows, Linux operating systems and IOS and Android phones.  

Find it here:

https://joplin.cozic.net/

brew install joplin 

The most important thing this project has nailed is cloud support and syncing.  I have my iPhone and computers syncing via Dropbox, which is easy to setup and works….  really well. Joplin folks have added many cloud options, so this is unlikely to be a sticking point for users.

Here are some of the key features:

  • Markdown is totally supported for straightforward and easy formatting
  • External editor support for emacs / atom / etc folks
  • Layout is clean, uncluttered, and just makes sense
  • Built-in markdown text editor and viewer is great
  • Notebook, todo, note, and tags work great across platforms
  • Browser integration, E2EE security, file attachments, and geolocation included

Hopefully this will be helpful.

Cheers,

- Jess

Mac OSX: Fixing GPT and PMBR Tables

February 27, 2019 / / 2 Comments

My computer recently crashed very, very hard, while I was removing an small empty alternative OS partition I no longer needed.  This is a fairly mundane operation that I do now and again, and is a ongoing fight to keep at least a few gigs of space free for actual work on precious 250gb Mac SSD.  

The crash results?  Toasted GPT tables all around.   My 2015 computer’s next move was to reboot- only to find essentially no partitions of memory… at all.  What it did show was (wait for it) Clover bootloader of all things, with a single windows boot camp icon (nothing in there either).  That is so wrong…. On all levels!

I brought the machine to the local university repair.  They declared this machine bricked and offered to wipe it.  Back to me it came…

I scheduled an Apple support session with a phone rep, which after around 45 minutes of actually productive troubleshooting ideas (none helping though) was forwarded to a senior supervisor.  She was interested in this problem, and we scheduled a larger block of time. But, in the meantime, I still wanted to try again….

How to recover a garbled GPT table for Mac OSX:

Start with clean SMC and PRAM / NVRAM.

Clearing these actually made accessing internet recovery (how we get to a stand-in OS with a terminal) dozens of times faster.  2.5 hours to 7 minutes. I actually waited 2.5 hours twice on separate attempts before I cleared these.

Follow these Apple links to perform these operations:

https://support.apple.com/en-us/HT204063

https://support.apple.com/en-us/HT201295

Get the computer with a text editor open.

Restart the computer into internet recovery.  Command + R or Command + Shift + R.

Wait.

Open a Terminal.  The graphical disk utility is useless because the disk / partition we want is unreachable(so it will say everything is great).

Run:

diskutil list

For me, I see disk0s2 is 180.6 gb.  That’s my stuff!

I also found /dev/disk2 → /dev/disk14 to be tiny partitions- don’t worry about those.

The syntax you are looking for is:

Name: “untitled” Identifier: disk#

(NOT disk#s#)

Write down ALL of the above information for the disk you are after.  That is probably disk0.

Then:

gpt -r show disk0

Copy the following readout in your terminal for all entries bigger than “32”.  The critical fields here are Start, Size, Index, and Contents. Each field is supremely important.

Here is mine (formatted for web):

# Disk0, with contents > “32” :

# First Table:

Start: 40  

Size: 409600

Index:  1

Contents: C12A7328-F81F-11D2-BA4B-00A0C93EC93B

# Second table, the one with my data:

Start: 409640

Size: 352637568

Index: 2

Contents: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF

Note, this is the initial Contents.  I rewrote this once with the correct Apple Index 2 data but did not create a new table (leaving the rest of the broken bits broken).  We are replacing / destroying a table here, but not the data.     

Actions:

# unmount the disk.  From here we are doing tables, not disks / data.

diskutil unmountDisk disk0

# Get rid of the GPT on the disk we are recovering.  We are not touching the data.

gpt destroy disk0

# Make a new one to start with some fresh values.

gpt create -f disk0

# perform magic trick

# USE THE DATA YOU WROTE DOWN FROM “gpt -r show disk0”.  THIS IS IMPORTANT.

# we must add that first small partition at index 1.  Verbatim.

gpt add -i 1 -b 40 -s 409600 -t C12A7328-F81F-11D2-BA4B-00A0C93EC93B disk0

# index two (for me) is my data.  We are going to use the default OSX / Mac HD partition values.

# the Length of “372637568” is not as sure fire as the GPT Contents.  

# YMMV, but YOLO.

gpt add -i 2 -b 409640 -s 372637568 -t 7C3457EF-0000-11AA-AA11-00306543ECAC disk0

Again, that Contents value is 7C3457EF-0000-11AA-AA11-00306543ECAC.

- Jess

written in the recovered computer xD

Musings On Chapel Language and Parallel Processing

February 20, 2019 / / 0 Comments

View below the readme mirror from my Github repo. Scroll down for my Python3 evaluation script.

....Or visit the page directly: https://github.com/Jesssullivan/ChapelTests 

ChapelTests

Investigating modern concurrent programming ideas with Chapel Language and Python 3

See here for dupe detection: /FileChecking-with-Chapel

Iterating through all files for custom tags / syntax: /GenericTagIterator

added 9/14/19:

The thinking here is one could write a global, shorthand / tag-based note manager making use of an efficient tag gathering tool like the example here. Gone are the days of actually needing a note manager- when the need presents itself, one could just add a calendar item, todo, etc with a global tag syntax.

The test uses $D for date: $D 09/14/19

//  Chapel-Language  //

// non-annotated file @ /GenericTagIterator/nScan.chpl //

use FileSystem;
use IO;
use Time;

config const V : bool=true;  // verbose logging, currently default!

module charMatches {
  var dates = {("")};  
}

// var sync1$ : sync bool=true;  not used in example- TODO: add sync$ var back in!!

proc charCheck(aFile, ref choice, sep, sepRange) {

    // note, reference argument (ref choice) is needed if using Chapel structure "module.domain"

    try {
        var line : string;
        var tmp = openreader(aFile);
        while(tmp.readline(line)) {
            if line.find(sep) > 0 {
                choice += line.split(sep)[sepRange];
                if V then writeln('adding '+ sep + ' ' + line.split(sep)[sepRange]);
            }
        }
    tmp.close();
    } catch {
      if V then writeln("caught err");
    }
}

coforall folder in walkdirs('check/') {
    for file in findfiles(folder) {
        charCheck(file, charMatches.dates, '$D ', 1..8);
    }
}

Get some Chapel:

In a (bash) shell, install Chapel:
Mac or Linux here, others refer to:

https://chapel-lang.org/docs/usingchapel/QUICKSTART.html

# For Linux bash:
git clone https://github.com/chapel-lang/chapel
tar xzf chapel-1.18.0.tar.gz
cd chapel-1.18.0
source util/setchplenv.bash
make
make check

#For Mac OSX bash:
# Just use homebrew
brew install chapel # :)

Get atom editor for Chapel Language support:

#Linux bash:
cd
sudo apt-get install atom
apm install language-chapel
# atom [yourfile.chpl]  # open/make a file with atom

# Mac OSX (download):
# https://github.com/atom/atom
# bash for Chapel language support
apm install language-chapel
# atom [yourfile.chpl]  # open/make a file with atom

Using the Chapel compiler

To compile with Chapel:

chpl MyFile.chpl # chpl command is self sufficient

# chpl one file class into another:

chpl -M classFile runFile.chpl

# to run a Chapel file:
./runFile

Now Some Python3 Evaluation:

# Ajacent to compiled FileCheck.chpl binary:

python3 Timer_FileCheck.py

Timer_FileCheck.py will loop FileCheck and find the average times it takes to complete, with a variety of additional arguments to toggle parallel and serial operation. The iterations are:

ListOptions = [Default, Serial_SE, Serial_SP, Serial_SE_SP]

  • Default - full parallel

  • Serial evaluation (--SE) but parallel domain creation

  • Serial domain creation (--SP) but parallel evaluation

  • Full serial (--SE --SP)

Output is saved as Time_FileCheck_Results.txt

  • Output is also logged after each of the (default 10) loops.

The idea is to evaluate a "--flag" -in this case, Serial or Parallel in FileCheck.chpl- to see of there are time benefits to parallel processing. In this case, there really are not any, because that program relies mostly on disk speed.

Evaluation Test:

# Time_FileCheck.py
#
# A WIP by Jess Sullivan
#
# evaluate average run speed of both serial and parallel versions
# of FileCheck.chpl  --  NOTE: coforall is used in both BY DEFAULT.
# This is to bypass the slow findfiles() method by dividing file searches
# by number of directories.

import subprocess
import time

File = "./FileCheck" # chapel to run

# default false, use for evaluation
SE = "--SE=true"

# default false, use for evaluation
SP = "--SP=true" # no coforall looping anywhere

# default true, make it false:
R = "--R=false"  #  do not let chapel compile a report per run

# default true, make it false:
T = "--T=false" # no internal chapel timers

# default true, make it false:
V = "--V=false"  #  use verbose logging?

# default is false
bug = "--debug=false"

Default = (File, R, T, V, bug) # default parallel operation
Serial_SE = (File, R, T, V, bug, SE)
Serial_SP = (File, R, T, V, bug, SP)
Serial_SE_SP = (File, R, T, V, bug, SP, SE)


ListOptions = [Default, Serial_SE, Serial_SP, Serial_SE_SP]

loopNum = 10 # iterations of each runTime for an average speed.

# setup output file
file = open("Time_FileCheck_Results.txt", "w")

file.write(str('eval ' + str(loopNum) + ' loops for ' + str(len(ListOptions)) + ' FileCheck Options' + "\n\\"))

def iterateWithArgs(loops, args, runTime):
    for l in range(loops):
        start = time.time()
        subprocess.run(args)
        end = time.time()
        runTime.append(end-start)

for option in ListOptions:
    runTime = []
    iterateWithArgs(loopNum, option, runTime)
    file.write("average runTime for FileCheck with "+ str(option) + "options is " + "\n\\")
    file.write(str(sum(runTime) / loopNum) +"\n\\")
    print("average runTime for FileCheck with " + str(option) + " options is " + "\n\\")
    print(str(sum(runTime) / loopNum) +"\n\\")

file.close()

Evaluating Ubuntu Pop OS: Dual Boot Setup

January 24, 2019 / / 0 Comments

Dual OS on a 2015 MacBook pro

As the costs of Apple computers continue to skyrocket and the price of useable amounts of storage zoom past a neighboring galaxy (for a college student at least), I am always on on the hunt for cost effective solutions to house and process big projects and large data.

Pop OS (a neatly wrapped Ubuntu) is the in-house OS from System76.  After looking through their catalog of incredible computers and servers, I thought it would be a good time to see how far I can go with an Ubuntu daily driver.  Of course, there are many major and do-not-pass-go downsides- see the below list:

  • Logic Pro X → There is no replacement 🙁   A killer DAW with fantastic AU libraries. I am versed with Reaper and Bitwig, but neither is as complete as Logic Pro.  I will be evaluating POP with an installation of Reaper, but with so few plugins (I own very few third party sets) this is not a fair replacement.
  • Adobe PS and LR:  I do not like Adobe, but these programs are... ...kind of crucial for most project of mine that involve 2d, raster graphics.  I continue to use Inkscape for many tasks, but it is irrelevant when it comes to pixel-based work and photo management / bulk operations.
  • AutoCAD / Fusion 360 / Sketchup:  I like FreeCAD a lot, but it is not at all like the other programs.  Not worse or better, but these are all very different animals for different uses.
  • Apple notes and other apple-y things:  OSX is extremely refined. Inter-device solutions are superb.  I have gotten myself used to Google Keep, but it is not quite at the in-house Apple level.
  • XCode and IOS Simulator environments:  I do use Expo, but frankly to make products for Apple you need a Mac.

Dual Boot (OSX and Pop Ubuntu) Installation on a 2015 MBP:

This process is quite simple, and only calls for a small handful of post-installation tweaks.  My intent is to create a small sandbox with minimal use of “extras” (no extra boot managers or anything like that)

Steps:

Partition separate “boot”, “home”, and other drives

  • I am using a 256gb micro sd partitioned in half for OSX and Pop_OS (Sandisk extreme, “v3” speed rating version card via a BaseQi slot adapter)

Use the partition tool in Mac disk utility.  Be sure to set these new partitions as FAT 32- we will be using ext4 and other more linux-y filesystems upon installation, so these need to be as generic as possible.

Get a copy of Pop_OS from System76.

Use Etcher (recommended) or any other image burning tool to create a boot key for Pop.  

The USB key only has one small job, in which Pop_os will be burned into a better location in your boot partition made in the previous step.  If you are coming from a hackintosh experience, fear not: everything will stay in the Macbook Pro, not extra USB safety dongles or Kexts, or Plist mods…!

BOOT INTO POP_OS:

Restart your computer and hold down the alt-option Key.  THIS IS HOW TO SWITCH from Pop_os, OSX, Bootcamp, and anything else you have in there.  You should see an “efi” option next to the default OSX. (note- at least in my case, the built-in bootloader defaults to the last used OS at each restart.)

Once you are in the Pop_OS installer, click through and select the appropriate partitions when prompted.  After this installation, you may remove the USB key and continue to select
“efi” in the bootloader.


ASSUMING ALL GOES WELL:

You are now in Pop_OS!  Using the alt/option key will become second nature… but some Pop key mappings may not.  Continue for a list of Macbook Pro - specific tweaks and notes.

First moves:

Go to the Pop Shop and get the “Tweaks” tool.  I made one or two small keymap changes, but this is likely personal preference.  

Default, important Key Mappings:

Command will act as a “control center-ish” thing.  It will not copy or paste anything for you.

Control does what Command did on OSX.  

Terminal uses Control+Shift for copy and paste, but only in Terminal:  if you pull a Control+Shift+C in Chrome, you will get the Dev tool GUI...  The Shift key thing is needed unless you are inclined to root around and change it.

Custom Boot Scripts and Services:

In an effort to make things simple, I made a shell script to house the processes I want running when I turn on the computer- this is to streamline the “.service” making process.  While it may only take marginally more time to make a new service, this way I can keep track of what is doing what from a file in my documents folder.

In terminal, go to where your services live if you want to look:

cd /etc/systemd/system

Or, cut to the chase:

sudo nano /etc/systemd/system/startsh.sh.service

Paste the following into this new file:

_____________Begin _After_This_Line____________________

[Unit]

Description=Start at Open plz

[Service]

ExecStart=/Documents/startsh.sh

[Install]

WantedBy=multi-user.target

_____________End _Above_This_Line____________________

Exit nano (saving as you go) and cd back to “/”.

cd

sudo nano /Documents/startsh.sh

Paste the following (and any scripts you may want, see the one I have commented out for odrive CLI) into this new file:

_____________Begin _After_This_Line____________________

#!/bin/bash

# Uncomment the following if you want 24/7 odrive in your system

# otherwise do whatever you want

#nohup "$HOME/.odrive-agent/bin/odriveagent" > /dev/null 2>&1 &

# end

_____________End _Above_This_Line____________________

After exiting the shell script, start it all up with the following:

sudo systemctl start startsh.sh

sudo systemctl enable startsh.sh

Cloud file management with Odrive CLI and Odrive Utilities:

Visit one of the two Odrive CLI pages- this one has linux in it:

https://forum.odrive.com/t/odrive-sync-agent-a-cli-scriptable-interface-for-odrives-progressive-sync-engine-for-linux-os-x-and-windows/499#linuxinst

Please visit this repo to get going with --recursive and other odrive utilities

https://github.com/amagliul/odrive-utilities


These are the two commands I ended up putting in a markdown file on my desktop for easy access.  Nope, not nearly as cool as it is on OSX. But it works…

Odrive sync: [-h] for help

```

python "$HOME/.odrive-agent/bin/odrive.py" sync

```

Odrive utilities:

```

python "$HOME/odrive-utilities/odrivecli.py" sync --recursive

```

Next, Get Some Apps:

Download Chrome.  Sign into Chrome to get your chrome OS apps loaded into the launcher- in my case, I needed Chrome remote desktop.  DO NOT DOWNLOAD ADDITIONAL PACKAGES for Chrome Remote Desktop, if that is your thing. They will halt all system tools (disk utils, Gnome terminal, graphical file viewer…   !!See this thread, it happened to me!! )

Stock up!  

Get Atom editor:  https://atom.io/

...Or my favorites: https://www.jetbrains.com/toolbox/app/

Rstudio:  https://www.rstudio.com/products/rstudio/download/#download

Mysql:  https://dev.mysql.com/downloads/mysql/

MySQL Workbench:  https://dev.mysql.com/downloads/workbench/

If you get stuck:  make sure you have tried installing as root ($ sudo su -) and verified passwords with ($ sudo mysql_secure_installation)  

See here to start “rooting around” MySQL issues:  https://stackoverflow.com/questions/50132282/problems-installing-mysql-in-ubuntu-18-04/50746032#50746032

Get some GIS tools:

QGIS!

sudo apt-get install qgis python-qgis qgis-plugin-grass

uGet for bulk USGS data download!

sudo add-apt-repository ppa:plushuang-tw/uget-stable

sudo apt install uget

That's all for now- Cheers!

-Jess

Deploy Shiny R apps along Node.JS

November 25, 2018 / / 0 Comments

Find the tools in action on Heroku as a node.js app!

https://kml-tools.herokuapp.com/

See the code on GitHub:

https://github.com/Jesssullivan/Shiny-Apps

After many iterations of ideas regarding deployment for a few research Shiny R apps, I am glad to say the current web-only setup is 100% free and simple to adapt.   I thought I'd go through some of the Node.JS bits I have been fussing with. 

The Current one:  

Heroku has a free tier for node.js apps.  See the pricing and limitations here: https://www.heroku.com/pricing as far as I can tell, there is little reason to read too far into a free plan; they don’t have my credit card, and thy seem to convert enough folks to paid customers to be nice enough to offer a free something to everyone.  

Shiny apps- https://www.shinyapps.io/- works straight from RStudio.  They have a free plan. Similar to Heroku, I can care too much about limitations as it is completely free.  

The reasons to use Node.JS (even if it just a jade/html wrapper) are numerous, though may not be completely obvious.  If nothing else, Heroku will serve it for free….

Using node is nice because you get all the web-layout-ux-ui stacks of stuff if you need them.  Clearly, I have not gone to many lengths to do that, but it is there.

Another big one is using node.js with Electron.  https://electronjs.org/ The idea is a desktop app framework serves up your node app to itself, via the chromium.  I had a bit of a foray with Electron- the node execa npm install execa package let me launch a shiny server from electron, wait a moment, then load a node/browser app that acts as a interface to the shiny process.  While this mostly worked, it is definitely overkill for my shiny stuff.  Good to have as a tool though.

-Jess

Recycled Personal “Cloud Computing” under NAT

November 5, 2018 / / 0 Comments

As many may intuit, I like the AWS ecosystem; it is easy to navigate and usually just works.  

...However- more than 1000 dollars later, I no longer use AWS for most things....

🙁   

My goals: 

Selective sync:  I need a unsync function for projects and files due to the tiny 256 SSD on my laptop (odrive is great, just not perfect for cloud computing.

Shared file system:  access files from Windows and OSX, locally and remote

Server must be headless, rebootable, and work remotely from under a heavy enterprise NAT (College)

Needs more than 8gb ram

Runs windows desktop remotely for gis applications, (OSX on my laptop)

 

Have as much shared file space as possible: 12TB+

 

Server:  recycled, remote, works-under-enterprise-NAT:

Recycled Dell 3010 with i5: https://www.plymouth.edu/webapp/itsurplus/

- Cost: $75 (+ ~$200 in windows 10 pro, inevitable license expense) 

+ free spare 16gb ram laying around, local SSD and 2TB HDD upgrades

- Does Microsoft-specific GIS bidding, can leave running without hampering productivity

Resilio (bittorrent) Selective sync: https://www.resilio.com/individuals/

- Cost: $60

- p2p Data management for remote storage + desktop

- Manages school NAT and port restrictions well (remote access via relay server)

Drobo 5c:

Attached and syncs to 10TB additional drobo raid storage, repurposed for NTFS

  • Instead of EBS (or S3)

 

What I see:  front end-

Jump VNC Fluid service: https://jumpdesktop.com/

- Cost: ~$30

- Super efficient Fluid protocol, clients include chrome OS and IOS,  (with mouse support!)

- Manages heavy NAT and port restrictions well

- GUI for everything, no tunneling around a CLI

  • Instead of Workspaces, EC2

Jetbrains development suite:  https://www.jetbrains.com/ (OSX)

- Cost:  FREE as a verified GitHub student user.

- PyCharm IDE, Webstorm IDE

  • Instead of Cloud 9

 

Total (extra) spent: ~$165

(Example:  my AWS bill for only October was $262)

 

-Jess

Quick fix: 254 character limit in ESRI Story Map?

September 12, 2018 / / 0 Comments

https://gis.stackexchange.com/questions/75092/maximum-length-of-text-fields-in-shapefile-and-geodatabase-formats

https://en.wikipedia.org/wiki/GeoJSON

https://gis.stackexchange.com/questions/92885/ogr2ogr-converting-kml-to-geojson

If you happened to be working with....  KML data (or any data with large description strings) and transitioning it into the ESRI Story Map toolset, there is a very good chance  you hit the the dBase 254 character length limit with the ESRI Shapefile upload.  Shapefiles are always a terrible idea.

 

the solution:  with GDAL or QGIS (alright, even in ArcMap), one can use GeoJSON as an output format AND import into the story map system- with complete long description strings!

 

QGIS:

Merge vector layers -> save to file -> GeoJSON

arcpy:
import arcpy

import os

arcpy.env.workspace = "/desktop/arcmapstuff"

arcpy.FeaturesToJSON_conversion(os.path.join("outgdb.gdb", "myfeatures"), "output.json")

GDAL:
<
ogr2ogr -f GeoJSON output.json input.kml

New App:  KML Search and Convert

July 27, 2018 / / 0 Comments

Written in R; using GDAL/EXPAT libraries on Ubuntu and hosted with AWS EC2.

New App:  KML Search and Convert

Here is an simple (beta) app of mine that converts KML files into Excel-friendly CSV documents.  It also has a search function, so you can download a subset of data that contains keywords.   🙂

The files will soon be available in Github.

I'm still working on a progress indicator; it currently lets you download before it is done processing.   Know a completely processed file is titled with "kml2csv_<yourfile>.csv".

...YMMV.  xD

GDAL for R Server on Ubuntu – KML Spatial Libraries and More

July 23, 2018 / / 2 Comments

GDAL for R Server on Red Hat Xenial Ubuntu - KML Spatial Libraries and More

If you made the (possible mistake) of running with a barebones Red Hat Linux instance, you will find it is missing many things you may want in R.   I rely on GDAL (the definitive Geospatial Data Abstraction Library) on my local OSX R setup, and want it on my server too.  GDAL contains many libraries you need to work with KML, RGDAL, and other spatial packages.  It is massive and usually take a long time to sort out on any machine.

These notes assume you are already involved with a R server (usually port 8787 in a browser).  I am running mine from an EC2 instance with AWS.

! Note this is a fresh server install, using Ubuntu; I messed up my original ones while trying to configure GDAL against conflicting packages. If you are creating a new one, opt for at least a T2 medium (or go bigger) and find the latest Ubuntu server AMI.  For these instructions, you want an OS that is as generic as possible.

On Github:

https://github.com/Jesssullivan/rhel-bits

From Bash:

# SSH into the EC2 instance: (here is the syntax just in case)

#ssh -i "/Users/YourSSHKey.pem" ec2-user@yourAWSinstance.amazonaws.com

sudo su -

apt-get update

apt-get upgrade

nano /etc/apt/sources.list

#enter as a new line at the bottom of the doc:

deb https://cloud.r-project.org/bin/linux/ubuntu xenial/

#exit nano

wget https://raw.githubusercontent.com/Jesssullivan/rhel-bits/master/xen-conf.sh

chmod 777 xen-conf.sh

./xen-conf.sh

Or...

From SSH:

# SSH into the EC2 instance: (here is the syntax just in case)

ssh -i "/Users/YourSSHKey.pem" ec2-user@yourAWSinstance.amazonaws.com

# if you can, become root and make some global users- these will be your access to

# RStudio Server and shiny too!

sudo su –

adduser <Jess>

# Follow the following prompts carefully to create the user

apt-get update

nano /etc/apt/sources.list

# enter as a new line at the bottom of the doc:

deb https://cloud.r-project.org/bin/linux/ubuntu xenial/

# exit nano

# Start, or try bash:

apt-get install r-base

apt-get install r-base-dev

apt-get update

apt-get upgrade

wget http://download.osgeo.org/gdal/2.3.1/gdal-2.3.1.tar.gz

tar xvf gdal-2.3.1.tar.gz

cd  gdal-2.3.1

# begin making GDAL: this all takes a while

./configure  [if your need proper kml support (like me), search on configuring with expat or libkml.   There are many more options for configuration based on other packages that can go here, and this is the step to get them in order...]

sudo make

sudo make install

cd # Try entering R now and check the version!

# Start installing RStudio server and Shiny

apt-get update

apt-get upgrade
sudo apt-get install gdebi-core
wget https://download2.rstudio.org/rstudio-server-1.1.456-amd64.deb
sudo gdebi rstudio-server-1.1.456-amd64.deb

# Enter R or go to the graphical R Studio installation in your browser

R

# Authenticate if using the graphical interface using the usr:pwd you defined earlier

# this will take a long time

install.packages(“rgdal”)

# Note any errors carefully!

Then:

install.packages(“dplyr”)

install.packages(c("data.table", "tidyverse”, “shiny”)  # etc

Well, there you have it!

-Jess

Extras:

##Later, ONLY IF you NEED Anaconda, FYI:

# Get Anaconda: this is a large package manager, and is could be used for patching up missing # dependencies:

#Use  "ls" followed by rm -r <anaconda> (fill in with ls results) to remove conflicting conda

# installers if you have any issue there, I am starting fresh:

mkdir binconda

# *making a weak attempt at sandboxing the massive new package manager installation*

cd binconda
wget http://repo.continuum.io/archive/Anaconda2-4.3.0-Linux-x86_64.sh
# install and follow the prompts
bash Anaconda2-5.2.0-Linux-x86_64.sh

# Close the terminal window completely and start a new one, and ssh back to where you left

# off.  Conda install requires this.

# open and SSH back into your instance.  You should now have either additional flexibility in

# either patching holes in dependencies, or created some large holes in your server.  YMMV.

### Done

Red Hat stuff:

Follow these AWS instructions if you are doing something else:

https://aws.amazon.com/blogs/big-data/running-r-on-aws/

See my notes on this here:

https://www.transscendsurvival.org/2018/03/08/how-to-make-a-aws-r-server/

and notes on Shiny server:

https://www.transscendsurvival.org/2018/07/16/deploy-a-shiny-web-app-in-r-using-aws-ec2-red-hat/

GDAL on Red Hat:- Existing threads on this:

https://gis.stackexchange.com/questions/120101/building-gdal-with-libkml-support/120103#120103

This is a nice short thread about building from source:

https://gis.stackexchange.com/questions/263495/how-to-install-gdal-on-centos-7-4

neat RPM package finding tool, just in case:

https://rpmfind.net/linux/rpm2html/

Info on the LIBKML driver if you end up with issues there:

http://www.gdal.org/drv_libkml.html

 

I hope this is useful- GDAL is important and best to set it up early.  It will be a pain, but so is losing work while trying to patch it in later.  xD

 

-Jess

 

INFO: Deploy a Shiny web app in R using AWS (EC2 Red Hat)

July 16, 2018 / / 0 Comments

Info on deploying a Shiny web app in R using AWS (EC2 Redhat)

As a follow-up to my post on how to create an AWS RStudio server, the next logical step is to host some useful apps you created in R for people to use.  A common way to do this is the R-specific tool Shiny, which is built in to RStudio.  Learning the syntax to convert R code into a Shiny app is rather subtle, and can be hard.  I plan to do a more thorough demo on this- particularly the use of the $ symbol, as in “input$output”- later. 🙂

 

It turns out hosting a Shiny Web app provides a large number of opportunities for things to go wrong….  I will share what worked for me.  All of this info is accessed via SSH, to the server running Shiny and RStudio.

 

I am using the AWS “Linux 2” AMI, which is based on the Red Hat OS.  For reference, here is some extremely important Red Hat CLI language worth being familiar with and debugging:

 

“sudo yum install” and “wget” are for fetching and installing things like shiny.  Don’t bother with instructions that include “apt-get install”, as they are for a different Linux OS!

 

“sudo chmod -R 777” is how you change your directory permissions for read, write, and execute (all of those enabled).  This is handy if your server disconnecting when the app tries to run something- it is a simple fix to a problem not always evident in the logs.  The default root folder from which shiny apps are hosted and run is “/srv/shiny-server” (or just “/srv” to be safe).

 

“nano /var/log/shiny-server.log” is the location of current shiny logs.

 

“sudo stop shiny-server” followed by “sudo start shiny-server” is the best way to restart the server- “sudo restart shiny-server” is not a sure bet on any other process.  It is true, other tools like a node.js server or nginx could impact the success of Shiny- If you think nginx is a problem, “cd /ect/nginx” followed by “ls” will get you in the right direction.  Others have cited problems with Red Hat not including the directories and files at “/etc/nginx/sites-available”.  You do not need these directories.  (though they are probably important for other things).

 

“sudo rm -r” is a good way to destroy things, like a mangled R studio installation.  Remember, it is easy enough to start again fresh!  🙂

 

“sudo nano /etc/shiny-server/shiny-server.conf” is how to access the config file for Shiny.  The fresh install version I used did not work!  There will be lots of excess in that file, much of which can causes issues in a bare-bones setup like mine.  One important key is to ensure Shiny is using a root user- see my example file below.  I am the root user here (jess)- change that to mirror- at least for the beginning- the user defined as root in your AWS installation.  See my notes HERE on that- that is defined in the advanced settings of the EC2 instance.

 

BEGIN CONFIG FILE:   (or click to download) *Download is properly indented


# Define user: this should be the same user as the AWS root user!
#
run_as jess;
#
# Define port and where the home (/) directory is
# Define site_dir/log_dir - these are the defaults
#
server{
listen 3838;
location / {
site_dir /srv/shiny-server;
log_dir /var/log/shiny-server;
directory_index on;
}
}

END CONFIG FILE

Well, the proof is in the pudding.   At least for now, you can access a basic app I made that cleans csv field data files that where entered into excel by hand.  They start full of missing fields and have a weird two-column setup for distance- the app cleans all these issues and returns a 4 column (from 5 column) csv.

Download the test file here:   2012_dirt_PCD-git

And access the app here:  Basic Shiny app on AWS!

Below is an iFrame into the app, just to show how very basic it is.  Give it a go!

-Jess

Off-Grid File Sharing with SAMBA / GL.iNet

June 25, 2018 / / 0 Comments

Note:  SMB / SharePoint is surely better with a proper server/computer.  A Raspberry Pi running OpenMediaVault (Debian) is a more common and robust option (still 5v low power).

If you are actually in an "it must done in OpenWRT" scenario, Click Here for my Samba config file: OpenWRT_Samba-config and see below.  Also, please use a NTFS or EX4 format.  🙂

 

...While my sharing method wasn't actually adopted by others, I still think it is good to know!

-Jess

How to Query KML point data as CSV using QGIS and R

June 10, 2018 / / 0 Comments

How to Query KML point data as CSV using QGIS and R

Here you can see more than 800 points, each describing an observation of an individual bird.  This data is in the form of KML, a sort of XML document from Google for spatial data.

 

I want to know which points have “pair” or “female” in the description text nodes using R.  This way, I can quickly make and update a .csv in Excel of only the paired birds (based on color bands).

 

 

Even if there was a description string search function in Google Earth Pro (or other organization-centric GIS/waypoint software), this method is more

robust, as I can work immediately with the output as a data frame in R, rather than a list of results.

 

First, open an instance of QGIS.  I am running ~2.8 on OSX.  Add a vector layer of your KML.

“Command-A” in the point dialog to select all before import!

Next, under “Vector”, select “Merge vector layers” via Data Management Tools.

 

Select CSV and elect to save the file instead of use a temporary/scratch file (this is a common error).

Open your csv in Excel for verification! 

 

 

 

 

 

 

The R bit:

# query for paired birds

#EDIT:  Libraries
library(data.table)
library(tidyverse)

data <- data.frame(fread("Bird_CSV.csv"))

pair_rows <- contains("pair", vars = data$description)

fem_rows <- contains("fem", vars = data$description)

result <- combine(pair_rows, fem_rows)

result <- data[result,]

write_csv(result, "Paired_Birds.csv")

Tada!

 

 

 

 

 

 

 

 

-Jess

Visual NH Research Update :)

June 8, 2018 / / 1 Comment

Olive-sided Flycatcher (Is Says, "Three Beers!! [please]"

Walking to work in the morning

Work

Song Sparrow

Common Yellowthroat Warbler

I'll let the photos to the talking:  welcome to my world! 🙂 !!!!

 

-Jess

 

 

Solar upgrades!

May 31, 2018 / / 0 Comments

Solar upgrades!

Incredibly, the hut we are working from actually had another solar panel just laying around.  🙂
This 50w square panel had a junction box with MC4 connectors, the standard for small scale solar installations.  As I was unsure how to know when we are running low on electricity reserves, I decided to make some adjustments.

Additional 50w solar panel

(Everything is still solder, hot glue, alligator clips, and zip-ties I’m afraid…)
I traded my NEMA / USA two-prong connection with two MC4 splitters, such that both panels can run in parallel (into a standard USA 110v extension cord that goes into our hut).  This way we should make well over one of the two 35ah batteries-worth of electricity a day.

Dual MC4 splitters to extension cord

I also added a cheap 12v battery level indicator.  It is not very accurate (as it fluctuates with solar input) but it does give us some insight about how much "juice" we have available.  (I also wired and glued the remote-on switch to the back of the input for stability.)

Added battery indicator and button

🙂
-Jess

Research Year Two: Three Photos

May 31, 2018 / / 0 Comments

Male Common Yellowthroat Warbler

The field season has officially started in Northern NH!

Male Common Yellowthroat warbler (COYE):   This fellow is defending a small territory in a patch of open thicket.   These warblers rely on early succession forest- patches of substrate that haven't  really grown in yet- to build cryptic, ground-level nests.  They develop complex systems to divert/confuse predators away from their nests.

 

Female Black-throated Blue Warbler (BTBW):  I was lucky to see this female.   She is paired with a male who defends a large mature forest territory.   They have quite a few BTBW neighbors, which makes for a lot of skirmishes among the males over land.  The females are often silent and move very fast...

Male Mourning Warbler (MOWA):  This is a rare bird here.   Even more amazing, it is defending a territory in our research site- and trying to chase out a male COYE while doing so.  The two species "share" resources, which means thy can't stand each other.   🙂   Each time the male COYE sings near the MOWA, it gets berated and chased away- and vice versa.   It appears the COYE isn't budging either, probably because it hasn't had this domestic, neighborly problem before.

-Jess

Female Black-throated Blue Warbler

Male Mourning Warbler

Gathering point data using Compass 55 on Apple iOS

May 28, 2018 / / 0 Comments

Keeping track of birds is tricky!

Attached is our team's workflow with Compass 55.    From the Kml, we go into Google Earth Pro - ArcGIS Desktop (arcmap).   QGIS is sometimes used too.

 

Cheers,

-Jess

 

840 Watts of Solar Power!

May 24, 2018 / / 0 Comments

Equipment used:

Inverter/PWM Controller:  http://a.co/fdl9YzI

2x 35ah Batteries: http://a.co/5JBIxTC

100w solar panel:  http://a.co/5JBIxTC

We need power!  While doing bird research in the wilds of northern NH, it became evident we needed electricity to power computers, big cameras, and phones/GPS units.

Below is a table of the system and our expected electricity needs:

System Solar 100w 35ah universal (x2)
Ah per day: 33.33333333 35 TOTAL Ah Reserve: 70
V 12 12 Parallel wiring: 12v
Wh in: 400 420 TOTAL Wh Reserve: 840
W 100
Cost $105.00 $64.00
ah/$ 2
Sun Hour / Multiplier 4 2
Need/Day Wh multiplier consump. in Wh = 259.36
Computer 100 2.5 250
iPhone 1.7 2 3.4
AAs 11.2 0.3 3.36
Camera 2.6 1 2.6

*The milk crate system below can charge a 100 watt MacBook Pro around 8-9 times from being completely empty.  

**Remember:  V*A=W,  W/V=A, and Watts over time is Wh.  

-Jess

+/- relates to size of standard prongs

Parallel maintains 12v but doubles Ah. (Series would go to 24v at 35ah)

 

Intro to the AWS Cloud 9 IDE

April 10, 2018 / / 0 Comments

The Cloud 9 IDE is the fastest way I have come up with to develop web-based or otherwise "connected" programs.    Because it lives on a Linux-based EC2 server on AWS, running different node, html, etc programs that rely on a network system just work- it is all already on a network anyway.   🙂  There is no downtime trying to figure out your WAMP, MAMP, Apache, or localhost situation.

Similarly, other network programs work just as well-  I am running a MySQL server over here (RDS), storage over there (S3), and have various bits in Github and locally.   Instead of configuring local editors, permissions, and computer ports and whatnot, you are modifying the VPC security policies and IAM groups- though generally, it just works.

Getting going:   The only prerequisite is you have an AWS account.  Students:  get $40 EC2 dollars below:

https://aws.amazon.com/education/awseducate/
Open the cloud 9 tab under services.

 

 

Setup is very fast- just know if others are going to be editing to, understand the IAM policies and what VPC settings you actually want.

 

Know this ideally a browser-based service; I have tried to come up with a reason a SSH connection would be better and didn't get any where.

For one person, micro is fine.   Know these virtual "RAMs" and "CPUs" are generous....

 

 

 

 

The default network settings are set up for you.   This follows good practice for one person; more than that (or if you are perhaps a far-travelling person) note these settings.  They are always editable under the VPC and EC2 instance tabs.

 

 

That's it!   Other use things to know:

This is a linux machine maintained by Amazon.   Packages you think should work and be up to date (arguably like any other linux machine I guess...)  may not be.  Check your basics like the NPM installer and versions of what your going to be working on, it very well may be different than what you are used to.

In the editor:

You have two panels of workspace in the middle- shown is node and HTML.   Everything is managed by tabs- all windows can have as much stuff as you want this way.

Below there is a "runner" (shown with all the default options!) and a terminal window.  Off to the left is a generic file manager.

 

 

I hope this is useful, it sure is great for me.

-Jess

Using ESRI ArcGIS / ArcMap in the AWS Cloud

April 5, 2018 / / 0 Comments

Selling AWS to... myself   🙂

Why struggle with underpowered local machines and VMs or watered-down web platforms for heavy lifting,  learning and work?

In addition to using ESRI software on mac computers, I am a big fan of the AWS WorkSpaces service (in addition to all their other developer tools, some of which are map-relevant: RDS for SQL and EC2 Redhat servers  for data management for example ).

Basically, for between ~$20 and ~$60 a month (Max, and not factoring in EDU discounts!), a user gets to use a well-oiled remote desktop.   You can download and license desktop apps like ArcMap and GIS products, file managers, and more from any computer connected to the internet.  This service is not very savvy; you make/receive a password and log right in.

A  big plus here of course is the Workspaces Application Manager (WAM); small sets of licenses can be administered in the same way desktops would, with extra easiness due to the "they are already really the same cloud thing any way".

Another plus is any client- netbook, macbook, VM, etc- will work equally well.  In this regard it can be a very cheap way to get big data work done on otherwise insufficient machines.  Local storage and file systems work well with the client application, with the caveat being network speed.

🙂

https://docs.aws.amazon.com/workspaces/latest/adminguide/amazon-workspaces.html

 

 

 

 

 

 

Using ESRI ArcGIS / ArcMap on Mac OSX: 2 methods

April 3, 2018 / / 2 Comments

Edit 07/26/2020:
Check out the expanded GIS notes page here!

Using ESRI ArcGIS / ArcMap on Macs: 2 methods

I need to run ESRI products on my MacBook Pro.   QGIS is always the prefered solution- open source, excellent free plugins, works on mac natively- but in a college / research environment, the only option that supports other people and school machines is ESRI.  Despite the annoying bureaucracy and expense of the software, some things are faster (but not better!) in ESRI, like dealing with raster / multiband data.

First, you need a license.

I went about this two ways;

My first solution was to buy an ESRI Press textbook on amazon.  A 180 day trial for $50- when taken as a college course, this isn't to bad.  🙂   The book is slow and recursive, but a 180 days to play with all the plugins and whistles allows for way deeper learning via the internet.   🙂

Do know there is a little-documented limit to the number of license transfers you may perform before getting either lock in or out of your software.  I hit this limit, as I was also figuring out my virtual machine situation, which would occasionally need a re-installation.

My current solution is “just buy a student license”.   $100 per year is less than any adobe situation- so really not that bad.  

Now you need a windows ISO.  

https://www.microsoft.com/en-us/software-download/windows10ISO

Follow that link for the window 10, 64 bit ISO.  YOU DO NOT NEED TO BUY WINDOWS.  It will sometimes complain about not having an  authentication, but in the months of using windows via VMs, never have I been prohibited to do... anything.  When prompted for a license when configuring your VM, click the button that says "I don't have a license".  Done.

 

Option one:  VirtualBox VM on a thumbdrive

https://www.virtualbox.org/wiki/Downloads - download for the VM software

http://a.co/4FEYMNY, http://a.co/hanHYl1 Suitable USBs.  the VM will take up most of a 128gb flash drive- ~70 gb just for windows and all the stuff you'll want from a PC.  Add ESRI software and allocated space for a cache (where your GIS project works!), bigger is better.   Format all drives in disk utility as ExFat!  this is important, any other file system either won't fly or could wreak havoc (other FAT based ones may have too small file allocations!

I used two drives, a 128 and a 64- this is great because I can store all my work on the 64, so I can easily plug it into other (school) machines running windows ArcMap and keep going, without causing issues with the massive VM in the 128.  

Installation is straightforward, just install EVERYTHING on the usb drive and it will be fine.   🙂

Problems:   Stability.   Crashes, and python / some other script modules do not work well.  This is a problem.  ArcAdministrator gets confused about all kinds of things- FWIW, if you are googling to delete the FLEXnet folder to solve authentication file issues, move to option 2 🙂

Speed is down, but actually the ~same speed as our school "super" PCs- (though I happened to know they are essentially glorified "hybrid" VMs too!) .

Option two: OSX Bootcamp 

https://support.apple.com/boot-camp

https://support.apple.com/en-us/HT201468

This way, you will hit "option/alt" each time you restart/boot your computer to choose from win/osx.   This is easy to install, as it is mac and mac = easy.

Big Caveat:  it is much harder to install windows externally  (on a usb, etc) from bootcamp.  I didn't succeed in my efforts, but there could be a way....   The thing is, it really wants to run everything like a normal intel based PC, with all installations in the usual place.  This is good for the mac performance, but terrible for the tiny SSD hard drives we get as mac users.  I have a 256gb SSD.  I have an average of < 15 gb wiggle room here, and use every cloud service in the book.

If you need to manage your cloud storage because of a itsy mac SSD, my solution is still ODrive.   https://www.odrive.com/

I use Amazon cloud mostly with odrive, but I use a personal/school OneDrives, Dropboxes, Google,  etc.  with only the occasional hiccup.   Also, all of the AWS tools are great and cheap- EC2, S3, Cloud 9, lambda, RDS.... Great way to do your work outside of your mac via the internet.

Result:

ArcMap and GIS stuff is blazing fast on my modest 2015 i5/8gb macbook pro.  Comparing a huge, mega ATX+ school computer to my mac on boot camp, I am running large raster filtering operations significantly quicker than other folks doing the same type of work.   That is GOOD.

🙂

-Jess

How to make a AWS R server

March 8, 2018 / / 1 Comment

When you need an R server and have lots of data to process, AWS is a great way to go.   Sign up of the free tier and poke around!

Creating an AWS Rstudio server:

https://aws.amazon.com/blogs/big-data/running-r-on-aws/ - using both the R snippet (works but the R core bits are NOT present and it will not work yet) and the JSON snippet provided  

https://www.rstudio.com/products/rstudio/download-server/ - the suite being installed

Follow most of the AWS blog AMI info, with the following items:

AMI:  Amazon Linux 2 (more packages and extras v. standard)  

  • t2.micro (free tier)
  • IAM policy follows AWS blog JSON snippet
  • Security Policy contains open inbound ports 22, 8787, 3838 (the latter two for R server specific communication)
  • Append user, username:password in the blog post’s initial r studio install text (pasted into the “advanced” text box when completing the AMI setup

 

SSH into the EC2 instance

sudo yum install –y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

sudo yum-config-manager --enable epel

sudo yum repolist

wget https://download2.rstudio.org/rstudio-server-rhel-1.1.423-x86_64.rpm

sudo yum update -y

sudo yum install -y R

sudo rstudio-server verify-installation

 

Access the graphical R server:

In a web browser, tack on “:8787” to the end of the Instance’s public “connect” link.  If it doesn’t load a login window (but seems to be trying to connect to something) the security policy is probably being overzealous……..

 

Notes on S3-hosted data:

  • S3 data is easiest to use if it is set to be public.
  • There are s3-specific tools for R, accessible as packages from CRAN directly from the R interface
  • Note data (delimited text at least) hosted in S3 will behave differently than it does locally, e.g. spaces, “na”, “null” need to be “cleaned” in R before use.  

 

There we have it!

 

-Jess

DIY CMoy headphone Amp, point to point: worth it?

December 22, 2017 / / 0 Comments

 

 

 

 

 

 

 

This is an extraordinarily simple headphone amp, and has essentially reached legendary/history status at this point.  I decided to build the original design (there are countless mods and totally different amps that hark back to this one), though opting for a wall-wart desk form factor instead of the original 9v battery tin.

 

I am building from the the 2008 "williamneo" blog post, as I like that point to point layout of his.  (I also could not easily procure the now ancient radioshack proto board- yes, the one shaped like a crab)  🙂

References:

http://williamneo.blogspot.com/2008/01/diy-cmoy-headphone-amplifier-for.html

https://tangentsoft.net/audio/cmoy/

 

opting for a desktop form-factor and 12 volts, my little amp works fine for easy to drive headphones.... But does it actually sound better than, say, an iphone?

 

The truth is, while it gets much louder than a phone, the OPA jfet op amp powering the whole thing is simply NOT high end.  it will distort with too much input (after a few clicks on a phone before the phone is maxed out for reference- well below standard line/"dac" level in both home and pro audio voltage wise) and will incrementally break up with hard/loud songs on bigger headphones, as the volume and rocking out goes up- in my case, the current fostex RP evolution I have been mangling is the big cheese candidate.  the planar drivers in the RP series are definitely "very hard" to drive in the scheme of things, but anything "heavy weight" will simply not do when pushed to an accessible limit.  I CAN use smaller dynamic/efficient  headphones with ease, such as my ported and open hifiman edition S without experiencing distortion.  Still here though, it is clear this amp is a "diy/cheap access to power"- slated against the fiio e12 for example, the CMOY seems a bit.... Loose?  Not any better, that is fore sure!  (granted, the e12 is a fantastic budget amp)

 

Note of op amps:   the NTE and other non-OPA brand-name look alikes and electrical analogues sound terrible.  I was fussing around with the NTE variants after needing the short out resistors "R5" (very important when building:  do not use R5) when they actually blew out from my using a inversely-poled power supply by accident, and was getting frustrated as the amp was working but sounded like a making TIDAL streaming gurgle out from a cheese grater.  eventually, after a ebay shipment of brand name OPAs came, I popped one in cautiously and it turned out the opamp was the source of the.... cheese grating.

In all, a good project for skill building but falls short of anything "hifi".

-Jess

My Fragile-Bodied Guide Scooter Safety

December 12, 2017 / / 0 Comments

I like scootering.  I like riding and learning tricks so much I think it it safe to say 94% of all my major injuries occur from skatepark mishaps….  Which is part of the reason I really don’t ride much anymore, because there is an extraordinarily good chance I am either still recovering from some injury from last time or am still certain I will surely break something again and am not ready for that extra burden of healing just yet.  🙂

Below is my (in process) Fragile-Bodied Guide Scooter Safety.

-Jess

Continue reading

Birding Beyond Binos: Find eBird Data for Fun!

August 25, 2017 / / 0 Comments

eBird is an indispensable tool for the modern, savvy birder.  It even rolls with my lingo on the home page:

Birding in the 21st Century!  What's not to like?

There are a number of key parts to the ebird experience as a viewer, only a few of which I will cover on this page.

The two most essential parts of ebird include the ability to learn about and enjoy birds when we can't get out into the field, and know where to go/what to expect when we can.

The first objective is fulfilled with the "Species Maps" button under the "Explore Data" tab.
This tool allows us to find and track birds and their friends (such as "migrant warblers") around the world.   This is a great way to get a handle on migrations, local owls (owls are great!) and other species that can otherwise "fly under your radar".

 

Here, with the species tab open, I tell eBird I want to know the whereabouts of the blackburnian warblers.  

 

 

Instead of showing me all the individual sightings ever, I specify this month range (Aug-Nov) and the last ten years option.

It turns out they are all over the place right now, having migrated up the eastern seaboard.    So...

...Using the next date range (you can be more specific than I) we can see where they migrate on the south-end of things.

 

Below:

Wow!!  they are all pretty much in South America, as our frosty winter has set in here in the north.  

As you may well imagine, this is an amazing tool to discover patterns and predict when birds may arrive and depart, letting us effectively, "bird from home".

Not a problem!

 

 

The second objective is equally simple with eBird: what can we expect from a location, bird wise?

 

Using the "Explore Hotspots" tab now, I can search for my local county.  YMMV on what criteria you will need, be it city, county, etc. 

All these little upside-down pears show up.  Similar to the first foray, we can specify dates from the dropdown tab- but really, just use the right hand button and specify "Past Week" or "Past Month" (as I have done here).   Now, the color code represents recent activity!   plan your trip and pack your bags!

I hope this is helpful breaking the ice into the world of eBird- it really fires up your birding lifestyle and ability!

-Jess

Gallery of “Warblers in the Hand”

July 27, 2017 / / 0 Comments

 

MPCNC: It moves!

July 25, 2017 / / 0 Comments

These are some photos of the current MPCNC project coming alive in the Sulliwood basement. The MPCNC is a (relatively) low cost, 3d printed CNC (computer numerical control=does stuff by itself)- featured here is the spindle (actually a drywall cutter) and a mashup mk8-style extruder. Below, you can see what I see before it is cutting time in the CAM module of Fusion 360. That object is half of a "trial run" pottery stand for someone's art show....

The MPCNC awakens!

Patchwork MPCNC extruder and hot end...

Double-sided MPCNC gcode, just for the holder part.....

 

...For this design:

Birding Beyond Binos: 5 Bird apps vs. “the Guide”.

July 3, 2017 / / 0 Comments

We all have a favorite bird, animal or plant guide.  Peterson is the best at drawing; Sibley takes the best pictures.  Kauffman ties it all together; National Geographic makes a solid reference and Audubon is great for fast looks.

While these books will always have a place on the shelf or table, the depth of content and portability of smartphone apps and trustworthy (e.g. reaserch related or associated with a big bird organization you recognize) websites truly foster the next level of ecological acuity.

[I will cover apps for iPhones and iPads- these are tools I have available and find to be indispensable.]  Like the shelf of guides they can replace out in the field, there is always room for another guide- and, generally speaking, cost significantly less than the least expensive print guide on your shelf.

  1. iBird PRO -

This app does it all: view photos, range maps, sounds, and similar birds, and search by band code, Latin/common name.  The sound recordings are pretty good and can be looped individually or as a species playlist (good for playback in research situations).  Similar bird songs are playable at the bottom of each species- great for learning and verifying nuances between similar songs.  The illustrations are “ok”- better than what I could do (obviously) but nothing quite like Peterson or Kauffman.  There are two more (add-on) engines in this app I have not used:  the local birds function by GPS (BAM) and a “humanized” search tool to pinpoint the bird you are looking for (Percevia).

  1. Audubon Birds

Audubon Birds has come a long way, and generally will offer more of a comprehensive written overview on each bird- going into feeding, behavior, breeding, and habitat discussions.  They seem to have added eBird integration (far, far superior to their “nature share” tool) which allows for both a mobile search into the unfathomably large user-based data set for local birds and a way to add your own data to eBird (though traditionally, the best way to do that is from a computer).

  1. Audubon Owls

This app is only a small vignette on owls; there seems to be more info geared solely about owls here than in Audubon Birds- photos, videos, tips, and tricks

  1. Merlin Bird ID

Despite the hardcore Bird Photo ID algorithms and location-based searches, this is geared toward those who may be starting out, and want to up the ante.  You fill in a few parameters about a bird sighting (this will not help with bird sounds), then it will generate a list of probable birds.  Supposedly, if you get a good photo of the bird on your phone (Digi scoping/Wi-Fi upload?) It can id the bird visually.

  1. eBird

If you are truly doing an eBird list for your trip, try this app for basic, quick additions- but I would not rely on it for media uploads or anything too crazy.  You can upload your checklist from the field then edit it later, though it is unclear if that is really a good idea in the scheme of data collection.

This is a list of the Bird apps I use on my phone, most getting use many times a week or even every day (iBird Pro).

-Jess

Rugby Morning #3.6

May 18, 2017 / / 0 Comments

...During #3 I got demolished by biting insects- ".6" times later I had purchased and applied a significant DEET and re-entered the fray!

-Jess

Rugby Morning #2

May 17, 2017 / / 0 Comments

Guess where I went this morning?

Breaking in the new spot.  Additionally, I saw Magnolia, Yellow, and Common Yellowthroat warblers, and heard Black Throated Blue and Green warblers.  Veery, Hermit, and  Ovenbird thrushes were around, in addition to catbirds.

-Jess

Some Environmental Rugby-Bird Portraits

May 12, 2017 / / 0 Comments

I scoped out the local "rugby" field this morning.  A retired birder-couple told me "188 Species" of birds have been spotted in the last decade (by them) in this mixed-habitat space.  Here's a start...

-Jess

 

Common Yellow Throat

Red Bellied Woodpecker

Hermit Thrush

Catbird

« Older posts Newer posts »

© 2021 Trans Scend Survival

Theme by Anders Noren — Up ↑