Code Snippets

• This python script is used to increase the number of replicas in an ES cluster and force those new replicas to be allocated to a new host to enable a filesystem backup of the ES data directory.

  	from requests import get, post, put
  	from json import dumps, loads
  	from sys import argv
  	from math import ceil
  	commands = []
  	def addShard(idx, shard):
  	commands.append({
  	"allocate" : {
  	"index" : idx,
  	"shard" : int(shard),
  	"allow_primary" : False,
  	"node" : backup_node
  	}
  	})
  	state = loads(get('http://localhost:9200/_cluster/state?filter_metadata=true').content)
  	data_nodes = []
  	for node in state['nodes']:
  	if "data" in state['nodes'][node]['attributes'] and state['nodes'][node]['attributes']['data'] == "false":
  	continue
  	data_nodes.append({ 'node': node, 'address': state['nodes'][node]['transport_address']})
  	replica_count = int(ceil(len(data_nodes)/2.0))
  	if len(argv) != 2:
  	print "You must specify which node is the backup node or the 'reset' command. Available nodes right now are:"
  	for node in data_nodes:
  	print "\t- %s (%s)" % (node['node'], node['address'])
  	quit()
  	if argv[1] == "reset":
  	put('http://localhost:9200/_settings', data='{ "index.routing.allocation.disable_allocation": false }')
  	put('http://localhost:9200/_settings', data='{ "index" : { "number_of_replicas" : %d } }' % replica_count)
  	quit()
  	backup_node = argv[1]
  	if backup_node not in state['nodes']:
  	print "Invalid data node"
  	quit()
  	put('http://localhost:9200/_settings', data='{ "index.routing.allocation.disable_allocation": true }')
  	put('http://localhost:9200/_settings', data='{ "index" : { "number_of_replicas" : %d } }' % (replica_count+1))
  	for idx in state['routing_table']['indices']:
  	for shard in state['routing_table']['indices'][idx]['shards']:
  	addShard(idx, shard)
  	# this is a lot of output, TODO: consolidate this into just what's useful
  	print dumps(loads(post('http://localhost:9200/_cluster/reroute?pretty', data=dumps({ "commands": commands })).content), indent=4)

  view raw es-backup-control.py hosted with ❤ by GitHub
• Mira Milk Frother Usage
  • To warm and froth milk:

  Push the on button once
  - blue indicator lights up
  

  • To only warm milk:

  Push the on button two time
  - blue indicator lights up (not helpful)
  

  • To only froth milk:

  Push the on button three times
  - the blue indicator flashes
  

  view raw mira-milk-frother-instructions.md hosted with ❤ by GitHub
• Backup your webtask code. Can be used to make sure you have the latest version of your code locally, or used to backup in case at some point webtask.io ceases to function. Considering they very recently started failing to install new npm modules, this could be any point now I fear.

  	import subprocess
  	from json import dumps
  	# NOTE: Files will be written to the current directory, without any checks to prevent overwriting. Archiving and/or version control are not considered here, but highly encouraged.
  	# get a list of all webtasks for the current user
  	list_request = subprocess.run(['wt', 'ls'], stdout=subprocess.PIPE, text=True)
  	# transform that output into a list of task names
  	tasks = [ ln.replace('Name:','').strip() for ln in list_request.stdout.split("\n") if 'Name:' in ln ]
  	for task in tasks:
  	print("processing", task)
  	# the output contains some meta information, might as well grab that while we're here
  	# it does include any npm modules configured so if you're looking to migrate to
  	# another service, host, etc this is useful information
  	meta = []
  	code = []
  	in_code = False
  	# grab the info for this specific webtask
  	task_request = subprocess.run(['wt', 'inspect', '--fetch-code', task], stdout=subprocess.PIPE, text=True)
  	# iterate through the lines of output splitting meta from code and stashing respectively
  	for line in task_request.stdout.split("\n"):
  	if in_code:
  	# we're past the meta section so this is all code now
  	code.append(line)
  	elif line == 'Code:':
  	# this is the beginning of the code section, we don't need this line
  	in_code = True
  	else:
  	# we're still in the header section so stash this line in meta
  	meta.append(line)
  	# write the meta file
  	with open("%s.meta" % task, 'w') as fp:
  	fp.write("\n".join(meta))
  	# write the code file
  	with open("%s.js" % task, 'w') as fp:
  	fp.write("\n".join(code))

  view raw webtask-backup.py hosted with ❤ by GitHub
• Accessing "member accounts" of your AWS organization. Since the AWS documentation and all the write-ups are more complicated than useful, there's the cliff notes.

  Switching to Child Account in AWS

  Prerequisites

  Setting up a child accocunt:

  • look at https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts_access.html#orgs_manage_accounts_create-cross-account-role
  • you only need steps 1-9

  Configuring switch roles

  This needs to be done for every user and needs to be done for every machine (stored in a cookie or local storage).

  Once you have switched from the parent account to a subaccount, your profile menu usually appears in the upper right corner will show the Display Name in a bubble colored with the chosen Color (below).

  Note: you can switch back to the parent org by selecting Back to [ YOUR USERNAME ] in that same profile menu.

  Preparation

  • Click on your username in the upper right corner to access your profile menu (left most of the menus in that corner)
  • Click on My Organization
  • In the Organizational Structure chart copy the account numbers and names of the subaccounts you wish to access

  For each subaccount you need access to follow these steps:

  • Make sure you are in the parent account (there will not be a Back to [ YOUR USERNAME ] option in the profile menu).
  • Open your profile menu by clicking on your username in the upper right corner (left most of the menus in that corner).
  • Select Switch role
  • If this is your first time doing this in the browser/computer you will need to click on Switch Role again
  • Enter the account details
    • Account: (as copied in Preparation above)
    • Role: OrganizationAccountAccessRole
    • Display Name: user's choice - whatever makes sense to you
    • Color: user's choice - NOTE: this will be the color of your username in the header when you're logged into this subaccount
  • Click Switch Role
  • Note you are now in the child account, as indicated by the visual change of the profile menu button in the upper right corner.

  view raw accessing child accounts in aws.md hosted with ❤ by GitHub
• Python port of the getContentHash() function from PHP's Composer. Useful when updating composer.lock from php when composer is available. Granted this "shouldn't be done" and is "doing it wrong", but sometimes running composer commands in your deployment chain is overkill and redundent.

  	import hashlib
  	from json import dumps
  	from collections import OrderedDict
  	# composerFileContents should be the dict version of the composer.json file
  	def getContentHash(composerFileContents):
  	relevant = OrderedDict((key, composerFileContents[key]) for key in composerFileContents.keys() if key in ['name','version','require','require-dev','conflict','replace','provide','minimum-stability','prefer-stable','repositories','extra'])
  	if 'config' in composerFileContents and 'platform' in composerFileContents['config']:
  	relevant['config']['platform'] = composerFileContents['config']['platform']
  	# hacky fix to cover the differences between JSON encoding in Python vs PHP
  	encstr = dumps(OrderedDict((key, relevant[key]) for key in sorted(relevant.keys())),separators=(',', ':')).replace("/", "\\/").encode()
  	return hashlib.md5(encstr).hexdigest()

  view raw getContentHash.py hosted with ❤ by GitHub

• 1
• 2
• 3
• 4
• 5
• 6