Long time no blog. During a recent OSINT investigation, I found that I needed to pull all domains found from my query on crt.sh. The problem I had, however, was that the results weren’t all that usable without a lot of copying, pasting, and cleaning.
To address this problem, and to save time in the future, I created a Jupyter Notebook to programmatically query the crt.sh website, dump the results into a pandas data frame (thinking that I’ll want to further enrich the data at a later date), and then printing out the unique list of results to the screen.
The code is written in Python 3, and relies on BeautifulSoup4, Pandas, and NumPy.
I’m calling it CrtShcrape (pronounced cert-shcrape) and you can download it from my GitHub here: https://github.com/andrewsmhay/CrtShcrape.
Hopefully, you can get some use from it. Until next time!
According to the National Oceanic and Atmospheric Administration (NOAA), a tornado (also called a twister, whirlwind, or cyclone) is a violently rotating column of air that extends from a thunderstorm and comes into contact with the ground. Tornado intensity is measured by the enhanced Fujita (EF) scale from 0 through 5, based on the amount and type of wind damage to a wide variety of structures ranging from trees to shopping malls.
The United States experiences more tornadoes than any other country in the world, especially in those states East of the Rocky Mountains. As a child, I always found myself wondering why people didn’t just move if they knew they were at risk of getting hit by a tornado. Of course, at the time, I had no sense of money, career, or family obligation to know that some people didn’t have the means to relocate. Without having a way to escape the danger, these people had to adapt their lifestyles to account for the unpredictable, and potentially devastating, weather.
This data alone makes me reconsider moving to an area constantly stricken by tornadoes.
Yet, there are people who want to help us better understand tornados, so that we can better prepare for them. In 1887, the first book on tornadoes was written by John Park Finley, a US Army Signal Service officer and pioneer in the field of tornado research. Finley’s book introduced the concept of a “tornado cave” that instructed readers to “get into it with your family and your treasures before the storm reaches you.” Furthermore, the book showed readers the plans for building their own “prize tornado cave” throughout several pages. The instructions included detailed architectural diagrams and even cost breakdowns for labor and materials – roughly USD 300 dollars, in case you were wondering.
While it was a revolutionary book containing many breakthrough ideas, it contained a few ideas which have since been proven false. One example that Finley wrote, “a tornado travels from southwest to northeast,” and, “if it is going to the right of you, run to the left” and vice versa. Based on his research at the time, this may have been accurate. Further research shows that tornadoes do not always travel from southwest to northeast.
While Finley was in the middle of his tornado research, the U.S. Army Signal Service banned the word “tornado” because they were concerned that word would cause panic. So, for more than half of a century, the weather reports ignored the word “tornado” and used the euphemisms – more on that later. One of Finley’s supporters, Edward S. Holden, tried to implement a tornado warning system using telegraph poles. But it was overshadowed by a report by Henry A. Hazen, a civilian employee of the corps, who deemed that because tornadoes were “exceedingly rare” and very localized, it was impossible to pinpoint forecasts.
From 1887 up until 1950, American weathermen were strictly forbidden to use the word “tornado” in the weather report. Back then, when science was still struggling to find a proper scientific explanation, they were considered a dark and mysterious force. In addition to upholding the “tornado” ban for decades, the Weather Bureau (which assumed jurisdiction from the Signal Corps in 1890) remained skeptical of the value and accuracy of tornado forecasts. It took until 1943 for experimental warning systems to be implemented; a public outcry in 1952 (after a severe outbreak that killed over 200 people) finally helped form the U.S. tornado research and forecasts.
Over the years, the storm cellar became the standard underground bunker design to protect the occupants from violent severe weather, such as tornadoes. The average storm cellar for a single-family was built close enough to the home to allow instant access in an emergency, but not so close that the house could tumble on the door during a storm, trapping the occupants inside. This was also the reason the main door on most storm cellars were mounted at an angle rather than perpendicular with the ground. An angled door allowed for debris to blow up and over the door, or sand to slide off, without blocking it, and the angle also reduced the force necessary to open the door if rubble had piled up on top.
In 1950, Congress simultaneously launched a system of nuclear bomb shelters and disaster relief for victims of natural disasters. It was then that the families living in tornado alley realized that these bomb shelters could serve a double purpose.
Research into improving buildings for resisting extreme winds began with the 1970 tornado in Lubbock, Texas. Twenty-six people were killed and about 1/3 of the city of 160,000 people was heavily damaged or destroyed. Texas Tech researchers produced a comprehensive documentary of building damage, the first of its kind. The concept of the above-ground storm shelter was presented in Civil Engineering magazine in 1974 by Texas Tech faculty member Dr. Ernst Kiesling and by Graduate Student David Goolsby. Intermittent development continued as available personnel and funding permitted.
As time passed, people started to ease up on their worry of being bombed, but the threat of tornadoes remained as common as the changing seasons. Since then, storm cellars or storm shelters have become a necessary part of life in many parts of the United States, and most people who do not own one are in search of one to go to during tornado season.
The total devastation of a small subdivision outside of Jarrell, TX in 1997 received national attention and news coverage, as did the widespread devastation of the Oklahoma City area on 3 May 1999. Many regional and local television companies and newspapers subsequently featured the above-ground storm shelter concept after severe storms struck this area.
Personnel of the Federal Emergency Management Agency (FEMA) observed the high level of interest in storm shelters among the public and published a prescriptive design booklet entitled, Taking Shelter from the Storm. The first edition was published in October 1998, the Second Edition in August 1999. After the events in Oklahoma City, FEMA and the state of Oklahoma put in place incentives for building storm shelters in houses that were being built or rebuilt after the tornado.
It wasn’t until June 2008 that a standard for the design and construction of storm shelters was approved.
In facing a life-threatening issue, we humans researched the problem, assessed the risk, and created mitigating controls to make the dangers of living in a tornado-rich environment tolerable. As time progressed, our ideas for mitigating controls spread to the masses and required additional research, guidance and eventually certification and accreditation to ensure the safety of its users.
Be safe out there and remember the words of comedian Ron White: “It’s not that the wind is blowing. It’s what the wind is blowing.”
If you’ve been in the information security field for at least a year, you’ve undoubtedly heard your organization defend the lack of investment in, change to or optimization of a cybersecurity policy, mitigating control or organizational belief. This “It hasn’t happened to us so it likely won’t happen” mentality is called optimism bias, and it’s an issue in our field that predates the field itself.