Category: starred

VC fund A16z’s Benedict Evans writes: We are in the middle of a wave of interesting new productivity software startups — there are dozens of companies that remix some combination of lists, tables, charts, tasks, notes, light-weight databases, forms, and some kind of collaboration, chat or information-sharing. All of these things are unbundling and rebundling spreadsheets, email and file shares. Instead of a flat grid of cells, a dumb list of files, and a dumb list of little text files (which is what email really is), we get some kind of richer canvas that mixes all of these together in ways that are native to the web and collaboration. Then, we have another new wave of productivity company that addresses a particular profession and bundles all of the tasks that were spread across spreadsheets, email and file shares into some new structured flow. […] A few years ago a consultant told me that for half of their jobs they told people using Excel to use a database, and for the other half they told people using a database to use Excel. There’s clearly a point in the life of any company where you should move from the list you made in a spreadsheet to the richer tools you can make in coolproductivityapp.io. But when that tool is managing a thousand people, you might want to move it into a dedicated service. After all, even Craigslist started as an actual email list and ended up moving to a database. But then, at a certain point, if that task is specific to your company and central to what you do, you might well end up unbundling Salesforce or SAP or whatever that vertical is and go back to the beginning. Of course, this is the cycle of life of enterprise software. IBM mainframes bundled the adding machines you see Jack Lemmon using below, and also bundled up filing cabinets and telephones. SAP unbundled IBM. But I’d suggest there are two specific sets of things that are happening now. First, every application category is getting rebuilt as a web application, allowing continuous development, deployment, version tracking and collaboration. As Frame.io (video!) and OnShape (3D CAD!) show, there’s almost no native PC application that can’t be rebuilt as a web app. In parallel, everything now has to be native to collaboration, and so the model of a binary file saved to a file share will generally go away over time (this could be done with a native PC app, but in practice generally won’t be). So, we have some generational changes, and that also tends to create new companies. But second, and much more important — everyone is online now. The reason we’re looking at nursing or truck drivers or oil workers is that an entire generation now grew up after the web, and grew up with smartphones, and assumes without question that every part of their life can be done with a smartphone. In 1999 hiring ‘roughnecks’ in a mobile app would have sounded absurd — now it sounds absurd if you’re not. And that means that a lot of tasks will get shifted into software that were never really in software at all before.

Read more of this story at Slashdot.

via Slashdot
We Are in the Middle of a Wave of Interesting New Productivity Software Startups

Link

We all know that Germans love David Hasselhoff, so it makes perfect sense that the Knight Rider and Baywatch star would appear in a commercial for a German automotive website. The words may be in German, but the humor is universal. Presumably, K.I.T.T. was replaced with a moped due to fuel efficiency concerns.

via The Awesomer
David Hasselhoff in Moped Rider

Transcript

CEO: I don’t understand why you are recommending blockchain for this application.
Boss: My staff are the experts, but I can explain the basic idea. You see, using blockchain is like losing a necklace on the beach. Then a seagull finds the necklace and takes it back to it’s nest. And we all like data security, don’t we?
CEO: It’s almost as if you are proposing a plan you don’t understand at any level.
Boss: Well, yes, but keep in mind that you wouldn’t understand it even if I could explain it.
CEO: But you’re sure someone on your staff understands it, right?
Boss: Define "sure".

via Dilbert Daily Strip
Comic for September 29, 2019

We reported earlier how Dianna Muller made national headlines, and arguably cemented a folk legend status in the firearms community, when she told the House Judiciary Committee “I will not comply” with an assault weapon ban. Di rocked for sure. But equally impressive, in a quite measured performance, Amy Swearer delivered the most important testimony of the entire hearing.

RELATED STORY

WATCH: Dianna Muller Tells House Judiciary Committee ‘I Will Not Comply’

Amy Swearer on the Assault Weapon Ban

During Swearer’s opening remarks, she read a prepared statement that showcased her deep knowledge of the subject. As a Senior Legal Policy Analyst with several published pieces, Swearer clearly conducted exhaustive research on assault rifles. She also has facts on crime statistics and the 2nd Amendment.

Then she simply schooled those in favor of an assault weapon ban.

Swearer gave a compelling argument that previously banned features make rifles safer, not more dangerous. She argued statistics show overwhelmingly that rifles don’t pose a great danger to society. Swearer showed that law-abiding citizens purchase these guns. And they use them for lawful purposes. Finally, Amy Swearer makes a point that AR-15s and the like give law-abiding citizens a chance against assailants.

Above all, Swearer showed the hypocrisy and fallacy of the anti-gun movement. Now let’s hope the politicians will listen.

1. ‘Assault Weapons’ Characteristics Make Them Safer, Not More Dangerous

“The term ‘assault weapon’ does not have one official definition, but typically denotes firearms that have a range of features associated with modern semi-automatic rifles such as the AR-15,” Amy Swearer read from her prepared statement. “It should be noted that the phrase ‘assault weapon’ is not a technical or legal term, but rather appears to have become popular as part of a concerted effort by gun control advocates to manipulate those with limited knowledge of firearms into confusing certain semi-automatic rifles with ‘assault rifles,’ which are functionally distinct and heavily regulated by the federal government.”

We all remember the inept restrictions of the previous AWB, limiting pistol grips, bayonet lugs, barrel shrouds and more. Swearer points out that gun control advocates typically attack such accessories and features. While they in no way change the functionality or legality of the firearm, they sure look scary. But in practice, a ban on these accessories takes away accessories that promote better marksmanship and better control. In effect, it makes the guns less safe.

“In short, proposals to ban ‘assault weapons’ are, for all intents and purposes, proposals to force law-abiding citizens to use firearms that are harder to fire accurately and more likely to cause them injuries, even when being used for lawful purposes.”

2. Semi-Automatic Rifles Not Significant Factor Behind Gun Violence

The attack on semi-automatic rifles shows an incredible level of hypocrisy from gun control advocates. Statistically speaking, AR-15s don’t constitute a problem. If this is really about safety and human life, the numbers point to handguns. However, the left knows that’s a much harder sell. With that in mind, they’ve demonized the black gun, painting it the mass murderer weapon of choice.

“Far from being the weapon of choice for would-be criminals, semi-automatic rifles are statistically the type of firearm least likely to be used for unlawful purposes, particularly compared to handguns,” Swearer said. “Over the last decade, rifles of any kind were definitively used in only 3-4 percent of gun homicides, and it is not clear how many of those deaths actually involved the use of ‘assault weapons’ compared to other types of rifles. The average American is, in fact, four times more lkely to be stabbed to death than he or she is to be shot to death with a rifle of any kind.

“Gun control advocates, politicians, and the media routinely characterize semi-automatic rifles, specifically the AR-15, as the ‘weapon of choice’ for mass public shooters,” Swearer continued. This is objectively incorrect. Over the last decade, more than half of mass public shooters have used handguns alone. Of those who did use rifles, the majority also brought other firearms, such as shotguns or handguns.”

3. Semi-Auto Rifles Commonly Owned by Law-Abiding Citizens, Used for Legitimate Purposes

The left likes to paint black gun ownership as a recent phenomenon. Additionally, gun control advocates often portray the AR-15 as a weapon of war. They say it’s used only to kill innocent civilians by mass murders.

“Over the last several decades, there has been a concerted effort by gun control activists to characterize certain semi-automatic rifles as ‘weapons of war’ that have ‘no business on our streets,’” Swearer said. “Ostensibly, this is to create the impression that the cosmetic features associated with firearms like the AR-15 serve no legitimate civilian purpose, and render a firearm objectively inappropriate for lawful uses like hunting, recreational target shooting, or self-defense. On its face, this is an absurd premise.”

“Moreover, federal law enforcement agencies refer to even select-fire AR-15 style rifles as ‘personal defense weapons,’ Swearer continued. “… It is little wonder, then, that many law-abiding citizens also rely on semi-automatic rifles as their own personal defense weapons, particularly in situations where law enforcement cannot protect the. Far from needing to be better protected from these rifles, law-abiding Americans benefit when they are allowed to defend themselves with them.”

4. ‘God Forbid’

To close, Swearer told the story of a woman who grew up uninitiated to firearms. She went decades before ever handling one, until her daughter took her to the range. They shot a handgun that day. The woman, according to Swearer, proved a terrible shotgun with a handgun from a mere few yards away. But then the woman picked up an AR-15, and she produced “a fist-sized grouping of lead in the center of that target from 20 yards out.”

That woman is Swearer’s mother.

“Now, I pray that my mother is never confronted with a situation where she is compelled to point a firearm at another human being, much less pull the trigger,” Swearer said. “I would infinitely prefer to live in a world where I never have to consider the possibility that someone would threaten her life or the lives of those around her.

“God forbid that my mother is ever faced with a scenario where she must stop a threat to her life,” Swearer continued. “But if she is, I hope she has a so-called ‘assault weapon’ to end that threat.”

The post VIDEO: Amy Swearer Gives 4 Points for No Assault Weapon Ban Before House Hearing appeared first on Personal Defense World.

via Personal Defense World
VIDEO: Amy Swearer Gives 4 Points for No Assault Weapon Ban Before House Hearing

Speculation & Opinion

USA – -(AmmoLand.com)- Most of us don’t want a second Civil War, though many of us will fight, and a very few even think war is a good idea. But the obvious question is what would it look like?

The answer is nothing like the last civil war. I polished off a bottle of McCallum 25 with a couple of old friends who think about this stuff for a living. You’d like these guys, bright, inventive; each kept up at night by their serious jobs.

The causes of the last Civil War were economic. Tariffs on products coming into and leaving southern ports meant the agrarian south was at odds with the more urban and industrialized north. No such division exists today. The left does not control states; the left controls cities; everything else belongs to the people.

The days of massed armies are long passed. So what can we expect?

Most liberal politicians will be in hiding or deceased within the 1st 72 hours. A nation of riflemen will step up.

If you picked a City, let’s say Chicago. Close off the roads, shut down the airports and the City has maybe five days of food, close off fuel or power and in the winter it freezes in two days.

Pick a City like Los Angeles that imports all its water, close that down and inside a week, the City is dry.

Civil unrest will follow – our major cities will burn from self-immolation.

Only in the big cities will the left control law enforcement. The State Guard units will not fire on civilians and will not necessarily come to defend cities where they do not live; they have families of their own to protect.

With 120,000,000 gun owners in America, 3% represents 3.6 million people. There are 850,000 sworn law officers in the US. The numbers favor those of us who stand for freedom. In the face of these number the 1st thing the left will try to do is forcefully disarm the people in the big cities. That may not go as smoothly as they plan.

The fact is the left is driving this. They have no tolerance for anyone who does not think and believe as they do. That type of division can only lead one direction. We must break their grip on the narrative.

ANTIFA is the left’s, Brown Shirts. When these masked cowards move to take power who will stand in their way? American gun owners will rally to protect their neighborhoods. We will fire on the ANTIFA crowds to protect our children. It would be a quick fight.

With my group, when the bottle got closer to empty, we asked some other questions:

• What happens when Mexico occupies Southern California and parts of Texas and Arizona to protect their citizens? What happens if they don’t stop there? With Gavin Newsom as the Governor of CA, will he welcome the Mexican invasion? Is the re-conquest of the lands lost in the Mexican/American war next?
• Can Canada handle the 10’s of millions of refugees?
• With our significant cities already being cesspools of disease and poverty, what happens afterward? How do you rebuild a country so divided with its major cities in ruins? Reconstruction will be harder than you realize.

So what can you do? Keep four weeks of food and water on hand. Buy a rifle. Pistols are excellent for personal defense, but there may be times when you need to reach out and extend your safe perimeter. Five hundred meters is right for me; your needs may vary. Keep enough ammunition and supplies to meet your family’s needs, then double it. Remember if your extended family knows your prepared you can expect guests.

Learn to store and handle your firearms safely and proficiency. (That is what REGULATED means in the 2nd Amendment.) Teach your children to shoot.

Get politically active, do what you can to stave off this conflict or prevent it.

So how else would it start? A President Booker throws gun owners in jail, or a President Sanders invited in blue helmets to help with firearms confiscation. Maybe it’s nothing more than a deranged kid with a grudge, like Gavrilo Princip.

Then again maybe it’s already stared and we’re just not paying attention.

Or it could be a plague. Thirty million undocumented illegal aliens huddled in barrios who shun the police and authorities are a potential biological timebomb. Combine that with an outbreak of Ebola, and a population scared of deportation it would give the disease time to spread. Would we see it 1st in our detention centers or on our streets? It would directly affect the Sanctuary City problem. Hispanics would take the brunt of the blame and be seen as carriers; the nation would begin to fall apart as moves to protect individual population groups unfold. Segregation would follow.

Remember, luck favors the well prepared. Have you thought long term about your family when it all goes south? I’d rather be prepared for something I hope never happens than left unprepared when if it does. How about you?

This article is a lesson for those on the left of thought; you cannot win.

The destruction you would invite is something to be avoided. Trump is NOT Hitler, in 2 or 6 years he will bow at the waist, thank us all for allowing him to serve; then he will walk away from power and enjoy his waning years. Be it Trump, Obama, Bush, or Clinton that is how it works. I hope you’re listening, so full of hate you on the left are that it blinds you to reality. Do not ring a bell you cannot un-ring.

I’m off to the range. Time to practice.

About Don McDougall

Don McDougall is an NRA instructor and member of the Los Padres “Friends of the NRA” committee. If he’s not at the range, you will find him setting the record straight with on gun issues and gun safety on AmmoLand Shooting Sports News.

The post War Is Coming!!! Are You Ready … Will You Fight? appeared first on AmmoLand.com.

via AmmoLand.com
War Is Coming!!! Are You Ready … Will You Fight?

I think Nintendo’s official Switch Pro Controller is brilliant. It’s a full-size pad with large face buttons, textured handles and a transparent shell that reveals a circuit-inspired pattern underneath. The accessory isn’t without its faults, though. The D-Pad, while a huge improvement over the left Joy-Con, is a bit stiff and spongy for my tastes. I also dislike the digital triggers, which barely depress, and the seriously steep $70 asking price.

The Switch has many third-party controllers to choose from, but none of them have impressed me like the 8BitDo SN30 Pro+. Terrible name aside, it’s an almost perfect pad that works not only with Nintendo’s hybrid console but Windows, macOS, Android and Raspberry Pi too.

Let’s start with the hardware. 8BitDo has rightfully earned a reputation for building modern, high quality controllers that ape those released for classic systems such as the NES and Sega Genesis. The SN30 Pro+ is clearly based on the iconic SNES controller, with a cross-shaped D-pad on the left, oblong start and select buttons in the middle, and circular A, B, X and Y inputs on the right-hand side. The "dogbone" design has been enhanced, however, with handles, analog sticks and two additional buttons — with a star and 8BitDo logo respectively — that, by default, act as the Home and Share shortcuts on the Switch.

It’s a stylish look that deftly balances Nintendo’s heritage with modern video game sensibilities. The controller also comes in three kickass color schemes: matte black, a SNES-themed grey and lavender, and a Gameboy-inspired cream and mulberry. I’ve been playing with the SNES variant but would happily own and display all three on my bookshelf.

Ergonomically, I prefer the SN30 Pro+ over the official Switch Pro Controller. The handles are longer and don’t dig into my palms as much during long play sessions. I like the symmetrical analog sticks, too, which ape Sony’s DualShock 4 — the other pad that I probably spend the most time with. The face buttons depress nicely and the D-Pad is practically identical to the one that shipped with the original Gameboy and SNES controller. It’s a simple, but proven design that works across a range of titles including Tetris 99, Street Fighter 30th Anniversary Collection and the entire Nintendo Switch Online library of NES and SNES games.

The SN30 Pro+ is lighter than the Switch Pro Controller, which worried me initially. The first time I picked it up, I was reminded of every cheap and horrifically made third-party controller I had to endure as a child. The SN30 Pro+ doesn’t flex, creak or generally feel like it’s about to disintegrate, though. My initial fears quickly dissipated, therefore, and I appreciated its scant weight whenever I threw it into my backpack.

Unlike the Switch Pro Controller, the SN30 Pro+ has analog triggers that know how far you’ve pressed them. That’s handy for select games like Trials Rising where you need to carefully ramp up your acceleration. I also appreciated the deeper pull distance in Fortnite and first-person shooters such as Wolfenstein II: The New Colossus. I don’t think the triggers improved my performance — I still haven’t won a game of Fortnite — but it made me more comfortable and, therefore, happier playing games competitively.

To my delight, the SN30 Pro+ has a removable battery pack. It lasts 20 hours on a single charge, which is only half the playtime advertised by the Switch Pro Controller. Still, I appreciated its flexibility. If you suddenly run out of juice and don’t have access to a power outlet, you can switch to AA batteries instead. It’s a useful backup for camping trips and long-haul flights that only offer a single USB port for charging your various gadgets.

You can wave the SN30 Pro+ around to perform basic motion controls. The experience is obviously inferior to a pair of Joy-Cons, but I didn’t have any problems waggling the pad to recharge my beam katana in Travis Strikes Again: No More Heroes. The SN30 Pro+ has rumble, too, though it’s not the special "HD Rumble" that Nintendo offers with its Joy-Cons and Switch Pro Controller. 8BitDo’s alternative also doesn’t have NFC — if you love and frequently use Amiibo to unlock extras on your Switch, that might be a dealbreaker.

To use the controller, you’ll need to go through the slightly long-winded syncing process. There are four different button combos that turn the pad on: Y and Start (Switch), B and Start (Android), X and Start (Windows), A and Start (macOS). Secondly, you’ll need to press and hold the pair button next to the USB-C port. The four LED lights at the bottom of the pad will then cycle to indicate that it’s hunting for a companion piece of hardware. Finally, on the Switch, you have to open the Change Grip/Order menu and, if required, hit the shoulder buttons. For all other platforms, the controller will appear as an available Bluetooth device.

It took awhile for the controller to sync up with my Switch. Thankfully, you only have to go through this process once on Nintendo’s system. For all subsequent play sessions, you can just press the Start button and the controller will instantly connect.

I was a tad disappointed that the SN30 Pro+ can’t turn on the Switch itself. It’s a small nitpick, but I love that with the Switch Pro Controller I can start playing Splatoon 2 or Super Mario Odyssey without standing up and walking over to my Switch Dock.

The SN30 Pro+ has one huge trick up its sleeve that makes up for almost all of its shortcomings: customization. Through the 8BitDo Ultimate Software, you can remap any button and create custom profiles. I created a simple one for the Switch, for instance, that swapped A and B around. (At long last, the face buttons line up with the ‘confirm’ and ‘cancel’ commands on my PS4.) Another window lets you change the ‘dead zone’ around each analog stick and the exact point at which a trigger registers your input. You can even set up macros that let you activate tricky combos with a single button press. (I shamefully used this to create a Hadouken shortcut.)

You can only switch profiles through the app, which is currently exclusive to Windows and Mac. For me, this wasn’t a problem — I used the controller’s default settings most of the time — but I can imagine how frustrating this would be for people that have game-specific profiles.

The SN30 Pro+ has a few other weaknesses. It doesn’t have a headphone jack like, for instance, PDP’s wired Faceoff Deluxe+ controller. I would have loved a simple carry case, too, to protect the pad on the road. And, at the time of writing, the pad doesn’t support PlayStation 4, Xbox One, iPhones and iPads, though 8BitDo has hinted that it’s working on support for iOS 13.

Otherwise, the SN30 Pro+ is phenomenal. It’s a true alternative to the Switch Pro Controller that makes sensible compromises. NFC and HD Rumble? I’ll happily sacrifice those for a better D-Pad and triggers. The customization options — while overkill for me — are brilliant and I love that I can also use the pad with my Android smartphone, MacBook Pro, and any gaming PC or Raspberry Pi-powered retro console I decide to build in the future. For $50, it’s a no-brainer. Even if you own a Switch Pro Controller, I would recommend buying one as your Player Two or Three pad. And, like me, you might find that you prefer it over Nintendo’s official offering.

Source: 8BitDo SN30 Pro+

via Engadget
8BitDo’s SN30 Pro+ is a near-perfect Switch controller

This describes four different methods for getting data into a Vue frontend from a Laravel application. From simply echo-ing data directly directly into components, to setting up jwt-auth and using JSON Web Tokens with axios to authenticate API calls.
via Laravel News Links
Four ways to pass data from Laravel to Vue

Managing the logs in a centralized repository is one of the most common best practices in the DevOps world. Application logs, system logs, error logs, and any databases logs also will be pushed into your centralized repository. You can use ELK stack or Splunk to visualize the logs to get better insights about it. But as a SQL guy, I wanted to solve this problem with Bigdata ecosystem(use SQL). As a part of that process, we can relationalize unstructured data in AWS Athena with the help of GrokSerDe.

Here S3 is my centralized repository. I know it will not scale like ElasticSearch, but why should I miss this Fun. For this use case, Im going to rationalize the SQL Server Error log in AWS Athena. Let’s take a look at the SQL server’s error log pattern.

2019-09-21 12:53:17.57 Server      UTC adjustment: 0:00
2019-09-21 12:53:17.57 Server      (c) Microsoft Corporation.
2019-09-21 12:53:17.57 Server      All rights reserved.
2019-09-21 12:53:17.57 Server      Server process ID is 4152.

Its looks like

yyyy-mm-dd space hh:mm:ss:ms space User space message

But sometimes, it has many lines like below.

2019-09-21 12:53:17.57 Server      Microsoft SQL Server 2017 (RTM) - 14.0.1000.169 (X64) 
  Aug 22 2017 17:04:49 
  Copyright (C) 2017 Microsoft Corporation
  Enterprise Edition: Core-based Licensing (64-bit) on Windows Server 2016 Datacenter 10.0 <X64> (Build 14393: ) (Hypervisor)
2019-09-21 12:53:17.57 Server      UTC adjustment: 0:00
2019-09-21 12:53:17.57 Server      (c) Microsoft Corporation.

If you see the 2nd, 3rd line we have the only message. And we know these all are just for information purpose, we’ll not get any useful information with that. Also as a part of Data cleansing, we should clean up some unwanted lines to make this relationalize.

I can consider the below format for my relationalize structure.

• Year – Integer
• Month – Integer
• Day – Integer
• Hour – Integer
• Minute – Integer
• Second – Integer
• User – String
• Message – String

We can convert this into a Grok pattern for this.

%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day}\\s*%{TIME:time} %{LOG_LEVEL:user}\\s*( )*%{GREEDYDATA:message}

Create the table in Athena:

CREATE EXTERNAL TABLE `sql_errorlog`(
  `year` string , 
  `month` string , 
  `day` string , 
  `time` string , 
  `user` string , 
  `message` string )  
ROW FORMAT SERDE 
  'com.amazonaws.glue.serde.GrokSerDe' 
WITH SERDEPROPERTIES ( 
  'input.format'='%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day}\\s*%{TIME:time} %{LOG_LEVEL:user}\\s*( )*%{GREEDYDATA:message}', 
'input.grokCustomPatterns'='LOG_LEVEL \[a-zA-Z0-9\]*') 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  's3://bhuvi-datalake/sql-error-log/'

The table got created. I used a custom pattern for pulling the user column.

Query the data:

SELECT *
FROM "default"."sql_errorlog" limit 10;

SELECT *
FROM "default"."sql_errorlog"
WHERE message LIKE '%shutdown%';

SELECT *
FROM "default"."sql_errorlog"
WHERE message LIKE '%Login failed%'

SELECT concat ('Server started at: ',year,'-',month,'-',day,' ',time) AS StartupTime
FROM "default"."sql_errorlog"
WHERE message LIKE '%Server process ID is%';

This is just a beginner guide, you can play around with windows logs, linux syslog, if you are a DBA then you may like to use this for MySQL, PostgreSQL, MongoDB logs.

BONUS: Regex Serde

If you are a developer, then regex might be easy for you. You can create a table with Regex Serde. Thanks to LeftJoin Who helped to write this Regex

CREATE EXTERNAL TABLE `bhuvi`(
  `date` string , 
  `time` string , 
  `user` string , 
  `message` string )
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.RegexSerDe' 
WITH SERDEPROPERTIES ( 
  'input.regex'='(.*\\-.*\\-.*)\\s+(\\d+:\\d+:\\d+.\\d+)\\s+(\\S+)\\s+(.*?)$') 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  's3://bhuvi-datalake/bhuvi-1/'

References:

1. Manage the SQL server workload with Customer Cloudwatch metrics.
2. AWS Athena Grok Serde.

via Planet MySQL
Relationalize Unstructured Data In AWS Athena with GrokSerDe

If you have been around MySQL for any length of time, you are probably aware that choosing the correct data types and optimizing your schema are actually important tasks.  A few years back at Percona Live 2016, I gave an introductory talk on schema review and optimization. Here’s the video:

I was thinking about that talk in the context of some of my current clients.  Though I had worked on extremely large database deployments during my earlier tenure at Percona, it was often more of an outlier.  Now, working as a Technical Account Manager with our largest clients, it is much more common.

The Fundamental Problem

I’d like to expand my thoughts on the “choosing the smallest data type you can” principle from my 2016 slides through the lens of a few of my 2019 clients.  I gave an example of two copies of the same table (a simple 4 column, 3 index table with ~4 million rows), one using a bigint for the primary key and one using a regular unsigned int for the primary key:

[root@sample-host plive_2016]# ls -alh action*ibd
-rw-rw---- 1 mysql mysql 908M Apr 7 16:22 action_bigint.ibd 
-rw-rw---- 1 mysql mysql 636M Apr 7 16:23 action.ibd

In this example, there was almost a 30% space savings associated with using the smaller data type.  Obviously, at the scale of 1GB of space, this is trivial. One comment I made during the talk references the adage “storage is cheap”.  While this can be true at a certain scale, I’m seeing this thinking break down more frequently with my largest clients.

The Problem Magnified at Scale

As an illustration, one of my clients is running roughly 10,000 Percona Server instances in their sharded deployment.  These servers are running on bare metal with above-average hardware (for performance concerns). While that sounds like a lot of servers (and it definitely is), you also have to take into consideration other operational concerns such as backups.  For the sake of some easier math, let’s assume the following:

• 5 servers/shard
• 500G data directory
• 5 backups of each shard (various time ranges such as daily, weekly, monthly)

Using those numbers as an estimate, one would be looking at roughly the following for space:

• ~4.7 petabytes storage for running instances (SSD)
• ~6 petabytes storage for backups (HDD)

The Business Impact on Large Deployments

Suddenly, at that scale, the 30% space that seemed trivial in my example seems a bit more important.  Let’s run some numbers based on the current $/TB pricing of SSD and spinning HDD with the 30% reduction in space:

• SSD Savings ~ $140,000
  • $100/TB Cost
  • 30% of 4.7PB = 1.4PB
• HDD Savings ~ $46,000

• • $25/TB Cost
  • 30% of 6PB = 1.8PB

Saving 3 petabytes of storage would reduce the raw storage costs by nearly $200,000.  On top of the actual capital costs of 3PB of storage, you have to consider server count, power, and maintenance (among others) which would increase this cost significantly.  Clearly, this is just a theoretical example of the potential impact a small mistake like picking the wrong data type can have on the overall infrastructure cost at scale. Generally, by the time a company reaches this scale, these sorts of optimizations have already been made and we need to start looking deeper and more creatively at other alternatives.

While this is definitely an oversimplification of calculating storage costs based on raw hardware prices, it does beg the question: even though it may not seem important now, why not ensure your schema is optimized before it becomes an expensive problem to fix!

For a more holistic view of how optimizations such as this one can save you money, check out our Database Management Savings Calculator or reach out to us for a more thorough review.

via Planet MySQL
Why Database Schema Optimization Matters

If you’re upgrading to a new iPhone from an old one, or from an Android device, then you need to take steps to make sure that Google Authenticator continues to work as you’d expect.
via AppleInsider
How to move Google Authenticator to your new iPhone