Crash reports are nearly impossible for most of us to understand, but that doesn’t mean you can’t make sense of them. Over on MacIssues, they point to a few things to look for in those crash reports on OS X so you can actually figure out the problem.
Obviously, there’s no single way to diagnose a problem, but there are some key words to look for. For example, “Abort” is a good term to search for to see why a crash happened, and from there you can pinpoint what the issue might be. Perhaps more important is the “Application Specific Information” section of a crash report, which is where you’ll get a slightly more specific diagnosis. In the MacIssues example, they found that a cache file was the culprit behind the crash, so it was pretty easy to troubleshoot the problem from there. If you have a program on OS X that’s crashing a lot and you simply can’t wait for the developer to respond to troubleshooting emails, being able to read that crash report is a useful skill. Head over to MacIssues for a nice overview of what you’ll want to look for.
Inked inscriptions from the fortress of Arad, located in southern Judah. Image: Michael Cordonsky / Tel Aviv University / Israel Antiquities Authority
Twenty six hundred years ago, a band of Judahite soldiers kept watch on their kingdom’s southern border in the final days before Jerusalem was sacked by Nebuchadnezzar. They left behind numerous inscriptions—and now, a groundbreaking digital analysis has revealed how many writers penned them. The research and innovative technology behind it stand to teach us about the origins of the Bible itself.
“It’s well understood that the Bible was not composed in real time but was probably written and edited later,” Arie Shaus, a mathematician at Tel Aviv University told Gizmodo. “The question is, when exactly?”
Shaus is one of several mathematicians and archaeologists trying to broach that question in a radical manner: by using machine learning tools to determine how many people were literate in ancient times. Their first major analysis, which appears today in the Proceedings of the National Academies of Sciences, suggests that the ability to read and write was widespread throughout the Kingdom of Judah, setting the stage for the compilation of Biblical texts.
Although parts of this conclusion remain controversial, the technology behind the study could revolutionize our understanding of literacy and education in Biblical times.
Most scholars agree that the earliest Biblical texts—including the Book of Joshua, Judges, and the two Books of Kings—took shape during what’s known as the late First Temple Period, before Jerusalem fell to the Babylonian king in 586 BCE. But the circumstances surrounding the writing of these texts, including when they were first penned and by how many authors, remain unclear. Curiously enough, texts that have nothing to do with the Bible may shed light on the matter.
Aerial view of the Tel arad fortress, where the inscriptions used in the present study originated. Image: Wikimedia
For instance, during this time period people wrote a wide variety of information down on ceramic pottery shards called ostraca. “These texts are very mundane in nature,” Shaus said, citing military commands and supply orders as some of the more popular topics of discussion.
Aside from how much wine Judahite soldiers required, however, there’s another layer of information we can extract from ostraca: how many people knew how to write. That’s exactly what Shaus and his colleagues did, analyzing a group of 16 well-preserved ceramic shards from a remote military fortress located near the southern border of Judah. Most of these ostraca date to around 600 BCE, practically the eve of the kingdom’s fall.
The first step of this analysis involved the researchers using novel image processing tools to restore characters that had been partially rubbed away. They then developed machine learning algorithms that could compare and contrast the shape of the ancient Hebrew characters in order to identify statistically distinct handwritings. In principle, this is similar to the algorithms tech companies use for digital signature detection.
“Handwriting analysis is a big area that’s seen a lot of research in recent years,” Shaus said. “Nevertheless, we had to develop our own tools and this was quite challenging. The medium is very deteriorated and so is the writing.”
Eventually, the team devised a handwriting recognition tool that worked beautifully on modern Hebrew, and they decided to put it to the test on ancient inscriptions. All in all, their analysis revealed at least six different authors behind the 16 ostraca. Examining the contents of the text itself, the researchers concluded that these authors spanned the entire military chain of command. “The commander down to the lowest water master could all communicate in writing,” Shaus said. “This was an extremely surprising result.”
It’s a result that the researchers say points to a “proliferation of literacy” throughout Judahite society by 600 BCE, implying that the educational infrastructure to support Bible writing almost certainly existed.
But not everyone is comfortable with all aspects of this conclusion.
“This is a highly innovative and important study,” Christopher Rollston, an expert on archaeology and Bible studies at George Washington University told Gizmodo, noting that there’s ample archaeological evidence portions of the Bible were written as early as 800 BCE. But who was really able to write at that time?
“I think that literacy was confined to elites, basically scribes, high military officials, and priests,” Rollston said, adding that by the late First Temple Period, it’s possible reading and writing had spread to more of this upper class.
Perhaps the most important aspect of Shaus’ work is the introduction of sophisticated image recognition technology to the study of ancient texts. The Tel Aviv research group is keen to share their tools for reconstructing letters and deciphering handwriting with other archaeologists. By applying these methods more broadly, we might be able to hone in on when, where, and by whom history’s most enduring book was first written down.
“We’re bringing new evidence to the game,” Shaus said. “Now, we’ll see what else comes out.”
This blog post will discuss the issues and solutions for MySQL Data at Rest encryption.
Data at Rest Encryption is not only a good-to-have feature, but it is also a requirement for HIPAA, PCI and other regulations.
There are three major ways to solve data encryption at rest:
Full-disk encryption
Database-level (table) encryption
Application-level encryption, where data is encrypted before being inserted into the database
I consider full disk encryption to be the weakest method, as it only protects from someone physically removing the disks from the server. Application-level encryption, on the other hand, is the best: it is the most flexible method with almost no overhead, and it also solves data in-flight encryption. Unfortunately, it is not always possible to change the application code to support application-level encryption, so database-level encryption can be a valuable alternative. Sergei Golubchik, Chief Architect at MariaDB, outlined the pluses and minuses of database level encryption during his
Sergei Golubchik, Chief Architect at MariaDB, outlined the pluses and minuses of database level encryption during his session at Percona Live Amsterdam:
Pros
Full power of DBMS is available
Full power of DBMS is availableEasy to implement
Easy to implementOnly database can see the data
Only databases can see the dataPer-table encryption, per-table keys, performance
Per-table encryption, per-table keys, performanceCannot be done per-user
Cons
Cannot be done per-user
Does not protect against malicious root user
Data at Rest Encryption: Database-Level Options
Currently, there are two options for data at rest encryption at the database level:
MariaDB 10.1.3+ support encryption (using Google patch)
MySQL 5.7.11+ (and Percona Server 5.7.11) has InnoDB tablespace level encryption
MariaDB’s implementation is different from MySQL 5.7.11. MySQL 5.7.11 only encrypts InnoDB tablespace(s), while MariaDB has an option to encrypt undo/redo logs, binary logs/relay logs, etc. However, there are some limitations (especially together with Galera Cluster):
No key rotation in the open source plugin version (MySQL 5.7.11 has a key rotation)
mysqlbinlog does not work with encrypted binlogs (bug reported)
Percona XtraBackup does not work, so we are limited to RSYNC as SST method for Galera Cluster, which is a blocking method (one node will not be available for writes during the SST). The latest Percona XtraBackup works with MySQL 5.7.11 tablespace encryption
The following data is not encrypted (bug reported)
Galera gcache + Galera replication data
General log / slow query log
Database level encryption also has its weakness:
Root and MySQL users can read the keyring file, which defeats the purpose. However, it is possible to place a key on the mounted drive and unmount it when MySQL starts (that can be scripted). The downside of this is that if MySQL crashes, it will not be restarted automatically without human intervention.
Both MariaDB version and MySQL version only encrypt data when writing to disk – data is not encrypted in RAM, so a root user can potentially attach to MySQL with gdb/strace or other tools and read the server memory. In addition, with gdb it is possible to change the root user password structure and then use mysqldump to copy data. Another potential method is to kill MySQL and start it with skip-grant-tables. However, if the key is unmounted (i.e., on USB drive), MySQL will either not start or will not be able to read the encrypted tablespace.
MariaDB Encryption Example
To enable the full level encryption we can add the following options to my.cnf:[mysqld]
plugin-load-add=file_key_management.so
file_key_management_filekey = FILE:/mount/keys/mysql.key
file-key-management-filename = /mount/keys/mysql.enc
innodb-encrypt-tables = ON
innodb-encrypt-log = 1
innodb-encryption-threads=1
encrypt-tmp-disk-tables=1
encrypt-tmp-files=1
encrypt-binlog=1
file_key_management_encryption_algorithm = AES_CTRAfter starting MariaDB with those settings, it will start encrypting the database in the background. The file_key_management plugin is used; unfortunately, it does not support key rotation. The actual keys are encrypted with:# openssl enc -aes-256-cbc -md sha1 -k <key> -in keys.txt -out mysql.encThe encryption <key> is placed in /mount/keys/mysql.key.
After starting MySQL, we can unmount the “/mount/key” partition. In this case, the key will not be available and a potential hacker will not be able to restart MySQL with “–skip-grant-tables” option (without passwords). However, it also prevents normal restarts, especially SSTs (cluster full sync).
Additional notes:
Encryption will affect the compression ratio, especially for the physical backups (logical backups, i.e. mysqldump does not matter as the data retrieved is not encrypted). If your original compressed backup size was only 10% of the database size, it will not be the case for the encrypted tables.
Data is not encrypted in flight and will not be encrypted on the replication slaves unless you enable the same options on the slaves. The encryption is also local to the server, so when encryption was just enabled on a server some tables may not be encrypted yet (but will be eventually)
To check which tables are encrypted, use the Information Schema INNODB_TABLESPACES_ENCRYPTION table, which contains encryption information. To find all tables that are encrypted, use this query:
select * from information_schema.INNODB_TABLESPACES_ENCRYPTION where ENCRYPTION_SCHEME=1
MySQL 5.7 Encryption Example
To enable encryption, add the following option to my.cnf:[mysqld]
early-plugin-load=keyring_file.so
keyring_file_data=/mount/mysql-keyring/keyringAgain, after starting MySQL we can unmount the “/mount/mysql-keyring/” partition.
To start encrypting the tables, we will need to run alter table table_name encryption=’Y’ , as MySQL will not encrypt tables by default.
The latest Percona Xtrabackup also supports encryption, and can backup encrypted tables.
To find all encrypted tablespaces in MySQL/Percona Server 5.7.11, we can use information_schema.INNODB_SYS_TABLESPACES and the flag field. For example, to find normally encrypted tables, use the following query:mysql> select * from information_schema.INNODB_SYS_TABLESPACES where flag = 8225G
*************************** 1. row ***************************
SPACE: 4688
NAME: test/t1
FLAG: 8225
FILE_FORMAT: Barracuda
ROW_FORMAT: Dynamic
PAGE_SIZE: 16384
ZIP_PAGE_SIZE: 0
SPACE_TYPE: Single
FS_BLOCK_SIZE: 4096
FILE_SIZE: 98304
ALLOCATED_SIZE: 98304
*************************** 2. row ***************************
SPACE: 4697
NAME: sbtest/sbtest1_enc
FLAG: 8225
FILE_FORMAT: Barracuda
ROW_FORMAT: Dynamic
PAGE_SIZE: 16384
ZIP_PAGE_SIZE: 0
SPACE_TYPE: Single
FS_BLOCK_SIZE: 4096
FILE_SIZE: 255852544
ALLOCATED_SIZE: 255856640
2 rows in set (0.00 sec)You can also use this query instead: select * from information_schema.tables where CREATE_OPTIONS like ‘%ENCRYPTION="Y"%’;.
Performance overhead
This is a debatable topic, especially for the MariaDB implementation when everything is configured to be encrypted. During my tests I’ve seen ~10% of overhead for the standalone MySQL instance, and ~20% with Galera Cluster.
The MySQL 5.7/Percona Server 5.7 tablespace-level encryption shows an extremely low overhead, however, that needs to be tested in different conditions.
Conclusion
Even with all the above limitations, database-level encryption can be a better option than the filesystem-level encryption if the application can not be changed. However, it is a new feature (especially MySQL 5.7.11 version) and I expect a number of bugs here.
via Planet MySQL MySQL Data at Rest Encryption
We’ve noted how Tennessee is one of twenty states that has passed state laws, quite literally written by companies like AT&T, prohibiting towns and cities from wiring themselves with broadband — even if nobody else will. When the FCC announced it would be taking aim at these protectionist broadband laws last year, Tennessee politicians threw a hissy fit, suing the FCC for "violating states rights." That incumbent ISPs are being allowed to write awful state law that’s hampering a generation of business development in the state? Not apparently much of a concern.
As an unsurprising result, Tennessee remains one of the least connected broadband states in the nation, and state residents have increasingly been giving beholden state politicians an earful. Those who can, like Tennessee developer John "Thunder" Thornton, have started taking matters into their own hands and building their own gigabit networks. To do so, Thorton had to get the aid of a power cooperative in Alabama, a state that has a slightly less restrictive municipal broadband law in place:
"Unable to gain high-speed broadband at what he deemed an affordable price from AT&T or Charter Communications and limited from service extensions from EPB’s ultra-fast Internet in Chattanooga, Thornton created his own Internet service provider last year. The private developer spent more than $400,000 to build his own fiber network and link it with a power cooperative in Stevenson, Ala., where fast broadband is available."
According to Thorton, the money he spent to wire his hilltop development cost a third of what AT&T was charging:
"Thornton said when he approached AT&T about providing Gig service to Jasper Highlands he was quoted a price of $1.3 million to serve his mountaintop development — more than three times what it ended up costing Thornton to build his own network connected to Alabama.
"Our costs are much less, but then I don’t have to pay for 27 lobbyists in Nashville like AT&T does," Thornton quipped.
So yeah, Tennessee lawmakers have done such a bang up job letting AT&T write awful state telecommunications law, state residents are being forced to spend their own money to get broadband at a relatively sane price. Sadly, most of the people that can’t get decent broadband can’t afford to go Thorton’s route. And while you’d think the cacophony of complaints from Tennessee residents would be enough to get some movement in the state legislature after a decade, all recent efforts to overturn the state’s protectionist law have gone nowhere.
Tennessee’s law prevents a popular Chattanooga-based utility-run ISP, EPB, from expanding its up to 10 Gbps offerings into any more markets. But attempts to repeal the law earlier this year went nowhere after mammoth pressure from incumbent ISP lobbyists. When that didn’t work, one lawmaker tried to pass a compromise bill that would have allowed EPB to expand into just one neighboring county. That proposal was shot down as well, one of the dissenting votes being that of Rep. Patsy Hazlewood, a former AT&T executive.
That leaves the FCC as the best, current option in getting some of these miserable laws overturned.
As it stands, the FCC is arguing that Section 706 of the Telecommunications Act allows it to preempt state laws that conflict with the agency’s Congressional mandate to guarantee "reasonable and timely" broadband deployment. While there’s twenty such laws, the FCC is currently trying to overturn just two: in Tennessee and North Carolina; the hope being the legal precedent then rolls downhill. But both states have taken the FCC to court, bravely defending their right to take campaign contributions — in exchange for protecting incumbent broadband providers from necessary and inevitable evolution.
There’s plenty of advice out there about how to make it in Silicon Valley. Some things are required for innovation success anywhere in the world, but the valley is a special place with different rules. As someone who’s spent a lot of time there, I think there are a few important elements to Silicon Valley’s success that others should know about if they’re trying to bring their ideas to life there.
Be Helpful: Successful People Help Others
You hear a lot these days about “corporate greed,” but even with so much money and prestige centered in Silicon Valley, the overall attitude of people there is very generous.…
via Phil McKinney What Is The Silicon Valley Secret Sauce For Innovation Success?
Non-Newtonian fluids are liquids that are also sort of solid but also sort of neither but also sort of both? They’re very thick liquids or very giving solids, depending on what you want to call it. That’s why they’re so fun to do science experiments with. The Backyard Scientist went and filled up balloons with non-Newtonian fluids and ripped a chainsaw through them, shot a BB-gun at them, and fired off a golf ball cannon at them too.
He wanted to compare how non-Newtonian fluid would react vs regular ol’ water in the same situation. Let’s just say the non-Newtonian fluid explosions were so much better (and they acted like a solid).