One of the better sites forfinancial data is Yahoo Finance. This makes it a prime target for web scraping by finance enthusiasts. There are nearly daily questions on StackOverflow that reference some sort of data retrieval (oftentimes through web scraping) from Yahoo Finance.
Web Scraping Problem #1
trying to test a code that scrap from yahoo finance
I’m a python beginner but I like to learn the language by testing it and trying it. so there is a yahoo web scraper…
stackoverflow.com
The OP is trying to find the current price for a specific stock, Facebook. Their code is below:
And that code produced the following output:
the current price: 216.08
It’s a pretty simple problem with an also simple web scraping solution. However, it’s not lazy enough. Let’s look at the next one.
Web Scraping Problem #2
Web Scraping Yahoo Finance Statistics — Code Errors Out on Empty Fields
I found this useful code snippet: Web scraping of Yahoo Finance statistics using BS4 I have simplified the code as per…
stackoverflow.com
The OP is trying to extract data from the statistics tab, the stock’s enterprise value and the number of shares short. His problem actually revolves around retrieving nested dictionary values that may or may not be there, but he seems to have found a better solution as far as retrieving data.
Take a look at line 3: the OP was able to find the data he’s looking for inside a variable in the javascript:
root.App.main = { .... };
From there, the data is retrieved pretty simply by accessing the appropriate nested keys within the dictionary, data. But, as you may have guessed, there is a simpler, lazier solution.
The lazy alternatives simply altered the request from utilizing the front-end URL to a somewhat unofficial API endpoint, which returns JSON data. It’s simpler and results in more data! What about speed though (pretty sure I promised simpler, more data, and a faster alternative)? Let’s check:
web scraping #1 min time is 0.5678426799999997
lazy #1 min time is 0.11238783999999953
web scraping #2 min time is 0.3731000199999997
lazy #2 min time is 0.0864451399999993
The lazy alternatives are 4x to 5x faster than their web scraping counterparts!
You might be thinking though, “That’s great, but where did you find those URLs?”.
The Lazy Process
Think about the two problems we walked through above: the OP’s we’re trying to retrieve the data after it had been loaded into the page. The lazier solutions went right to the source of the data and didn’t bother with the front-end page at all. This is an important distinction and, I think, a good approach whenever you’re trying to extract data from a website.
Step 1: Examine XHR Requests
An XHR (XMLHttpRequest) object is an API available to web browser scripting languages such as JavaScript. It is used to send HTTP or HTTPs requests to a web server and load the server response data back into the script. Basically, it allows the client to retrieve data from a URL without having to do a full page refresh.
I’ll be using Chrome for the following demonstrations, but other browsers will have similar functionality.
Open Chrome’s developer console. To open the developer console in Google Chrome, open the Chrome Menu in the upper-right-hand corner of the browser window and select More Tools > Developer Tools. You can also use the shortcut Option + ⌘ + J (on macOS), or Shift + CTRL + J (on Windows/Linux).
Select the “Network” tab
Then filter the results by “XHR”
Your results will be similar but not the same. You should notice though that there are a few requests that contain “AAPL”. Let’s start by investigating those. Click on one of the links in the left-most column that contain the characters “AAPL”.
After selecting one of the links, you’ll see an additional screen that provides details into the request you selected. The first tab, Headers, provides details into the request made by the browser and the response from the server. Immediately, you should notice the Request URL in the Headers tab is very similar to what was provided in the lazy solutions above. Seems like we’re on the right track.
If you select the Preview tab, you’ll see the data returned from the server.
Perfect! It looks like we just found the URL to get OHLC data for Apple!
Step 2: Search
Now that we’ve found some of the XHR requests that are made via the browser, let’s search the javascript files to see if we can find any more information. The commonalities I’ve found with the URLs relevant to the XHR requests are “query1” and “query2”. In the top-right corner of the developer’s console, select the three vertical dots and then select “Search” in the dropdown.
Search for “query2” in the search bar:
Select the first option. An additional tab will pop-up containing where “query2” was found. You should notice something similar here as well:
It’s the same variable that web scraping solution #2 targeted to extract their data. The console should give you an option to “pretty-print” the variable. You can either select that option or copy and paste the entire line (line 11 above) into something like https://beautifier.io/ or if you use vscode, download the Beautify extension and it will do the same thing. Once it’s formatted appropriately, paste the entire code into a text editor or something similar and search for “query2” again. You should find one result inside something called “ServicePlugin”. That section contains the URLs that Yahoo Finance utilizes to populate data in their pages. The following is taken right out of that section:
This is the same URL that is utilized in the lazy solutions provided above.
TL;DR
While web scraping can be necessary because of how a website is structured, it’s worth the effort investigating to see if you can find the source of the data. The resulting code is simpler and more data is extracted faster.
Finding the source of a website’s data is often found by searching through XHR requests or by searching through the site’s javascript files utilizing your browser’s developer console.
More Information
What if you can’t find any XHR requests? Check out The Alternative to Web Scraping, Part II: The DRY approach to retrieving web data
The Alternative to Web Scraping, Part II
The DRY approach to retrieving web data
towardsdatascience.com
If you’re interested specifically in the Yahoo Finance aspect of this article, I’ve written a python package, yahooquery, that exposes most of those endpoints in a convenient interface. I’ve also written an introductory article that describes how to use the package as well as a comparison to a similar one.
The (Unofficial) Yahoo Finance API
A Python interface to endless amounts of data
towardsdatascience.com
Please feel free to reach out if you have any questions or comments
A team of researchers from Beihang University, the Peking University School and Hospital of Stomatology and the Michigan Institute of Translational Nanotechnology has developed a synthetic enamel with properties similar to natural tooth enamel. In their paper published in the journal Science, the group describes their enamel and how well it compared to natural enamel when tested.
[…]
Prior research has shown that the reason that human enamel is so strong and yet also slightly elastic is because it consists of tiny rods made of calcium that are packed tightly together like pencils in a box. In their new effort, the researchers attempted to mimic tooth enamel as closely as possible by producing a material using AIP-coated hydroxyapatite nanowires that were aligned in parallel using a freezing technique that involved applying polyvinyl alcohol.
The researchers applied the enamel to a variety of shapes, including human teeth, and then tested how well it performed. They found it had a high degree of stiffness, was strong and was also slightly elastic. They also found that on most of their tests, the synthetic enamel outperformed natural enamel.
The researchers plan to keep testing their material to make sure it will hold up under harsh environments such as those found in the human mouth. They will also have to show that it is safe for use in humans and that it can be mass produced. They note that if their enamel passes all such tests, it could be used in more than just dentistry—they suggest it could be used to coat pacemakers, for example, or to shore up bones that have been damaged or that have eroded due to use or disease.
In December 2021,27,591 aircraft took off or landed at Frankfurt airport—890 every day. But this winter, many of them weren’t carrying any passengers at all. Lufthansa, Germany’s national airline, which is based in Frankfurt, has admitted to running 21,000 empty flights this winter, using its own planes and those of its Belgian subsidiary, Brussels Airlines, in an attempt to keep hold of airport slots.
Although anti-air travel campaigners believe ghost flights are a widespread issue that airlines don’t publicly disclose, Lufthansa is so far the only airline to go public about its own figures. In January, climate activist Greta Thunberg tweeted her disbelief over the scale of the issue. Unusually, she was joined by voices within the industry. One of them was Lufthansa’s own chief executive, Carsten Spohr, who said the journeys were “empty, unnecessary flights just to secure our landing and takeoff rights.” But the company argues that it can’t change its approach: Those ghost flights are happening because airlines are required to conduct a certain proportion of their planned flights in order to keep slots at high-trafficked airports.
A Greenpeace analysis indicates that if Lufthansa’s practice of operating no-passenger flights were replicated equally across the European aviation sector, it would mean that more than 100,000 “ghost flights” were operating in Europe this year, spitting out carbon dioxide emissions equivalent to 1.4 million gas-guzzling cars. “We’re in a climate crisis, and the transport sector has the fastest-growing emissions in the EU,” says Greenpeace spokesperson Herwig Schuster. “Pointless, polluting ‘ghost flights’ are just the tip of the iceberg.”
Aviation analysts are split on the scale of the ghost flight problem. Some believe the issue has been overhyped and is likely not more prevalent than the few airlines that have admitted to operating them. Others say there are likely tens of thousands of such flights operating—with their carriers declining to say anything because of the PR blowback.
“The only reason we have [airport] slots is that it recognizes a shortage of capacity at an airport,” says John Strickland of JLS Consulting, an aviation consultant. “If there wasn’t any shortage of capacity, airlines could land and take off within reason whenever they want to.” However, a disparity between the volume of demand for takeoff and landing slots and the number of slots available at key airports means that airlines compete fiercely for spaces. In 2020, 62 million flights took place at the world’s airports, according to industry body Airports Council International. While that number sounds enormous, it’s down nearly 40 percent year on year. To handle demand, more than 200 airports worldwide operate some kind of slot system, handling a combined 1.5 billion passengers. If you board a flight anywhere in the world, there’s a 43 percent chance your flight is slot managed.
Airlines even pay their competitors to take over slots: Two highly prized slots at London Heathrow airport reportedly changed hands for $75 million in 2016, when tiny cash-rich airline Oman Air made Air France-KLM an offer it couldn’t refuse for a sleepy 5.30 am arrival from Muscat to the UK capital.
One leading developer described the move as ‘vile,’ while another said Apple is deliberately ensuring it would cost developers more to opt-out of Apple’s payment system than it would to remain within it …
Background
Dutch regulators, like those in South Korea, ordered that Apple allow developers to opt-out of the App Store payment platform. Apple initially said that it would comply, but didn’t give any details.
The company today announced that it would reduce its commission by only three percent for those who chose to do so, and would also impose onerous administrative overheads – such as applying for permission to use a specific API, maintaining a separate version of the app, and filing reports with Apple.
[…]
Marco Arment highlighted the conditions imposed by Apple:
Separate app, only available in Netherlands
Cannot also support IAP
Must display scary sheets before payment
Website links are all to a single URL specified in Info.plist with no parameters
Must submit monthly report to Apple listing EVERY external transaction
Adding:
And after you pay your ~3% to your payment processor, Apple’s 27% commission takes you right back up to 30%. Glorious. Come on, THIS is comedy. Amazing, ridiculous comedy. I’d be surprised if a single app ever took them up on this. (And that’s exactly by design.)
Crisis Text Line, one of the nation’s largest nonprofit support options for the suicidal, is in some hot water. A Politico report last week highlighted how the company has been caught collecting and monetizing the data of callers… to create and market customer service software. More specifically, Crisis Text Line says it “anonymizes” some user and interaction data (ranging from the frequency certain words are used, to the type of distress users are experiencing) and sells it to a for-profit partner named Loris.ai. Crisis Text Line has a minority stake in Loris.ai, and gets a cut of their revenues in exchange.
As we’ve seen in countless privacy scandals before this one, the idea that this data is “anonymized” is once again held up as some kind of get out of jail free card:
“Crisis Text Line says any data it shares with that company, Loris.ai, has been wholly “anonymized,” stripped of any details that could be used to identify people who contacted the helpline in distress. Both entities say their goal is to improve the world — in Loris’ case, by making “customer support more human, empathetic, and scalable.”
But as we’ve noted more times than I can count, “anonymized” is effectively a meaningless term in the privacy realm. Study after study after study has shown that it’s relatively trivial to identify a user’s “anonymized” footprint when that data is combined with a variety of other datasets. For a long time the press couldn’t be bothered to point this out, something that’s thankfully starting to change.
For the past two weeks, observers of North Korea’s strange and tightly restricted corner of the internet began to notice that the country seemed to be dealing with some serious connectivity problems. On several different days, practically all of its websites—the notoriously isolated nation only has a few dozen—intermittently dropped offline en masse, from the booking site for its Air Koryo airline to Naenara, a page that serves as the official portal for dictator Kim Jong-un’s government. At least one of the central routers that allow access to the country’s networks appeared at one point to be paralyzed, crippling the Hermit Kingdom’s digital connections to the outside world.
[…]
But responsibility for North Korea’s ongoing internet outages doesn’t lie with US Cyber Command or any other state-sponsored hacking agency. In fact, it was the work of one American man in a T-shirt, pajama pants, and slippers, sitting in his living room night after night, watching Alien movies and eating spicy corn snacks—and periodically walking over to his home office to check on the progress of the programs he was running to disrupt the internet of an entire country.
Just over a year ago, an independent hacker who goes by the handle P4x was himself hacked by North Korean spies. P4x was just one victim of a hacking campaign that targeted Western security researchers with the apparent aim of stealing their hacking tools and details about software vulnerabilities. He says he managed to prevent those hackers from swiping anything of value from him. But he nonetheless felt deeply unnerved by state-sponsored hackers targeting him personally—and by the lack of any visible response from the US government.
So after a year of letting his resentment simmer, P4x has taken matters into his own hands. “It felt like the right thing to do here. If they don’t see we have teeth, it’s just going to keep coming,” says the hacker. (P4x spoke to WIRED and shared screen recordings to verify his responsibility for the attacks but declined to use his real name for fear of prosecution or retaliation.)
[…]
P4x says he’s found numerous known but unpatched vulnerabilities in North Korean systems that have allowed him to singlehandedly launch “denial-of-service” attacks on the servers and routers the country’s few internet-connected networks depend on.
[…]
he named, as an example, a known bug in the web server software NginX that mishandles certain HTTP headers, allowing the servers that run the software to be overwhelmed and knocked offline. He also alluded to finding “ancient” versions of the web server software Apache,
[…]
“It’s pretty interesting how easy it was to actually have some effect in there.”
[…]
He acknowledges that his attacks amount to no more than “tearing down government banners or defacing buildings,” as he puts it. But he also says that his hacking has so far focused on testing and probing to find vulnerabilities. He now intends to try actually hacking into North Korean systems, he says, to steal information and share it with experts. At the same time, he’s hoping to recruit more hacktivists to his cause with a dark website he launched Monday called the FUNK Project—i.e. “FU North Korea”—in the hopes of generating more collective firepower.
[…]
he was nonetheless shocked and appalled by the realization that he’d been personally targeted by North Korea.
P4x says he was later contacted by the FBI but was never offered any real help to assess the damage from North Korea’s hacking or to protect himself in the future. Nor did he ever hear of any consequences for the hackers who targeted him, an open investigation into them, or even a formal recognition from a US agency that North Korea was responsible. It began to feel, as he put it, like “there’s really nobody on our side.”
[…]
While he acknowledges that his attacks likely violate US computer fraud and hacking laws, he argues he hasn’t done anything ethically wrong. “My conscience is clear,” he says.
After a years-long process, data protection officials across the European Union have ruled that Europe’s ad tech industry has been operating unlawfully. The decision, handed down by Belgium’s APD (.PDF) and agreed by regulators across the EU, found that the system underpinning the industry violated a number of principles of the General Data Protection Regulations (GDPR). The Irish Council for Civil Liberties has declared victory in its protracted battle against the authority which administers much of the advertising industry on the continent: IAB Europe.
At the heart of this story is the use of the Transparency and Consent Framework (TCF), a standardized process to enable publishers to sell ad-space on their websites. This framework, set by IAB Europe, is meant to provide legal cover — in the form of those consent pop-ups which blight websites — enabling a silent, digital auction system known-as Real-Time Bidding (RTB). But both the nature of the consent given when you click a pop-up, and the data collected as part of the RTB process have now been deemed to violate the GDPR, which governs privacy rights in the bloc.
The APD has ruled that any and all data collected as part of this Real-Time Bidding process must now be deleted. T
[…]
Regulators have also handed down an initial fine of €250,000 to IAB Europe and ordered the body to effectively rebuild the ad-tech framework it currently uses. This includes making the system GDPR compliant (if such a thing is possible) and appoint a dedicated Data Protection Officer.
Hackers stole more than $324 million in cryptocurrency from Wormhole, the developers behind the popular blockchain bridge confirmed Wednesday.
The platform provides a connection that allows for the transfer of cryptocurrency between different decentralized-finance blockchain networks. Wormhole said in a series of tweets Wednesday afternoon that thieves made off with 120,000 wETH, or wrapped ethereum, worth nearly $324 million at current exchange rates. The platform’s network was also taken offline for maintenance.
according to a new FAQ posted on Google’s Workplace administrator forum. At the end of that month, the company will be adding a new feature—“Workspace search history”—that can continue to track these customers, even if they, or their admins, turn activity tracking off.
The worst part? Unlike Google’s activity trackers that are politely defaulted to “off” for all users, this new Workplace-specific feature will be defaulted to “on,” across Workspace apps like Gmail, Google Drive, Google Meet, and more.
[…]
Luckily, they can turn this option off if they want to, the same way they could turn off activity settings until now. According to Google, the option to do so will be right on the “My Activity” page once the feature goes live, right alongside the current options to flip off Google’s ability to keep tabs on their web activity, location history, and YouTube history. On this page, Google says the option to turn off Workspace history will be located on the far lefthand side, under the “Other Google Activity” tab.
Intel Corporation no longer has to pay a €1.06bn ($1.2bn, £890m) fine imposed by the European Commission (EC) in 2009 for abusing its dominance of the chip market.
On Wednesday, the General Court of the European Union annulled the EC antitrust penalty [PDF] after previously upholding it in 2014 [PDF].
After rival AMD complained in 2000 and again in 2003 that Intel was engaging in anti-competitive conduct by offering its hardware partners rebates for using Intel’s x86 chips, an EC antitrust investigation that got underway in 2004 and concluded in 2009 with a €1.06 billion penalty against Chipzilla.
The EC at the time found Intel’s conduct between October 2002 and December 2007 to be anti-competitive.
“The evidence gathered by the Commission led to the conclusion that Intel’s conditional rebates and payments induced the loyalty of key OEMs and of a major retailer, the effects of which were complementary in that they significantly diminished competitors’ ability to compete on the merits of their x86 CPUs,” the EC said in its 2009 decision. “Intel’s anti-competitive conduct thereby resulted in a reduction of consumer choice and in lower incentives to innovate.”
[…]
The ruling suggests that EU trustbusters won’t be able to constrain corporate behavior if alleged misconduct fails to fit within the limited definition of competitive abuse under EU law (Article 102 TFEU). According to the Associated Press, EC VP Margrethe Vestager said at a press briefing in Brussels that the EC needs more time to consider what comes next.
there are plenty of cases where you throw down hundreds of dollars for a piece of hardware and then you end up being the product anyway. Case in point: TVs.
On Wednesday, the television giant LG announced a new offering to advertisers that promises to be able to reach the company’s millions of connected devices in households across the country, pummeling TV viewers with—you guessed it—targeted ads. While ads playing on your connected TV might not be anything new, some of the metrics the company plans to hand over to advertisers include targeting viewers by specific demographics, for example, or being able to tie a TV ad view to someone’s in-store purchase down the line.
If you swap out a TV screen for a computer screen, the kind of microtargeting that LG’s offering doesn’t sound any different than what a company like Facebook or Google would offer. That’s kind of the point.
[…]
Aside from being an eyesore that literally no TV user wants, these ads come bundled with their own privacy issues, too. While the kinds of invasive tracking and targeting that regularly happens with the ads on your Facebook feed or Google search results are built off of more than a decade’s worth of infrastructure, those in the connected television (or so-called “CTV”) space are clearly catching up, and catching up fast. Aside from what LG’s offering, there are other players in adtech right now that offer ways to connect your in-app activity to what you watch on TV, or the billboards you walk by with what you watch on TV. For whatever reason, this sort of tech largely sidesteps the kinds of privacy snafus that regulators are trying to wrap their heads around right now—regulations like CPRA and GDPR are largely designed to handle your data is handled on the web, not on TV.
[…]
The good news is that you have some sort of refuge from this ad-ridden hell, though it does take a few extra steps. If you own a smart TV, you can simply not connect it to the internet and use another device—an ad-free set-top box like an Apple TV, for instance—to access apps. Sure, a smart TV is dead simple to use, but the privacy trade-offs might wind up being too great.
OpenSea has revealed just how much of the NFT activity on its platform is defined by fakery and theft, and it’s a lot. In fact, according to the company, nearly all of the NFTs created for free on its platform are either spam or plagiarized.
The revelation began with some drama. On Thursday, popular NFT marketplace OpenSea announced that it would limit how many times a user could create (or “mint”) an NFT for free on the platform using its tools to 50. So-called “lazy minting” on the site lets users skip paying a blockchain gas fee when they create an NFT on OpenSea (with the buyer eventually paying the fee at the time of sale), so it’s a popular option especially for people who don’t have deep pockets to jumpstart their digital art empire.
This decision set off a firestorm, with some projects complaining that this was an out-of-the-blue roadblock for them as they still needed to mint NFTs but suddenly couldn’t. Shortly after, OpenSea reversed course and announced that it would remove the limit, as well as provided some reasoning for the limit in the first place: The free minting tool is being used almost exclusively for the purposes of fraud or spam.
Finland’s government says the mobile devices of its diplomats have been hacked using Pegasus spyware.
The Finnish foreign ministry stated on Friday that some of its officials abroad had been targeted by the sophisticated software.
“The highly sophisticated malware has infected users’ Apple or Android telephones without their noticing and without any action from the user’s part,” the Foreign Ministry said in a statement.
“Through the spyware, the perpetrators may have been able to harvest data from the device and exploit its features.”
[…]
NSO says it only sells Pegasus to governments for the purpose of fighting crime and terrorism.
But an investigation last year revealed that the spyware had been used to target journalists, activists and politicians in a number of countries — including France, Spain, and Hungary.
A recent Citizen Lab report also found that critics of Poland’s right-wing government were hacked using Pegasus.
Chinese satellite was observed grabbing another satellite and pulling it out of its normal geosynchronous orbit and into a “super-graveyard drift orbit.” The maneuver raises questions about the potential applications of these types of satellites designed to maneuver close to other satellites for inspection or manipulation and adds to growing concerns about China’s space program overall.
On January 22, China’s Shijian-21 satellite, or SJ-21, disappeared from its regular position in orbit during daylight hours when observations were difficult to make with optical telescopes. SJ-21 was then observed executing a “large maneuver” to bring it closely alongside another satellite, a dead BeiDou Navigation System satellite. SJ-21 then pulled the dead satellite out of its normal geosynchronous orbit and placed it a few hundred miles away in what is known as a graveyard orbit. These distant orbits are designated for defunct satellites at the end of their lives and are intended to reduce the risk of collision with operational assets.
The unusual maneuver was observed by telescopes belonging to commercial space awareness firm Exoanalytic Solutions. During a webinar hosted by the Center for Strategic and International Studies (CSIS) this week, Exoanalytic Solutions’ Brien Flewelling said the SJ-21 satellite “appears to be functioning as a space tug.” Space Command did not respond to a request for comment, Breaking Defense reports.
Space Force has been increasingly turning to commercial space companies to provide a variety of data and services to boost its situational awareness, and to that end, Joint Task Force-Space Defense awarded Exoanalytic Solutions a contract in 2021 to provide space domain data. “Comms, data relay, remote sensing, and even ISR and some other things — [these] capabilities are increasingly available in the commercial market,” Space Force deputy Lt. Gen. David Thompson said last year.
SJ-21, or Shijian-21, was launched in October 2021 atop a Long March-3B rocket. The satellite is officially designated as an On-Orbit Servicing, Assembly, and Manufacturing, or OSAM satellite, a broad class of satellites designed with capabilities to get close to and interact with other satellites. Such systems could enable a wide range of applications including extending the life of existing satellites, assembling satellites in orbit, or performing other maintenance and repairs. According to Chinese state news outlets, SJ-21 was designed to “test and verify space debris mitigation technologies.”
No personal data is sent to our server in any way. Nothing. Zilch. Nada. All the face detection algorithms will run on your own computer, in the browser.
In this ‘test’ your face is compared with that of all the other people who came before you. At the end of the show you can, if you want to, share some anonimized data. That will then be used to re-calculate the new average. That anonymous data is not shared any further.
Engineers at the University of Illinois Chicago have built a cost-effective artificial leaf that can capture carbon dioxide at rates 100 times better than current systems. Unlike other carbon capture systems, which work in labs with pure carbon dioxide from pressurized tanks, this artificial leaf works in the real world. It captures carbon dioxide from more diluted sources, like air and flue gas produced by coal-fired power plants, and releases it for use as fuel and other materials.
[..]
Illustration of a carbon capture process designed by UIC College of Engineering scientists. Carbon dioxide from air or flue gas is absorbed by a dry organic solution to form bicarbonate ions, which migrate across a membrane and are dissolved in a liquid solution to concentrated CO2. Carbon atoms are shown in red, oxygen atoms are shown in blue and hydrogen atoms are shown in white. (Credit: Aditya Prajapati/UIC)
Using a previously reported theoretical concept, the scientists modified a standard artificial leaf system with inexpensive materials to include a water gradient — a dry side and a wet side — across an electrically charged membrane.
On the dry side, an organic solvent attaches to available carbon dioxide to produce a concentration of bicarbonate, or baking soda, on the membrane. As bicarbonate builds, these negatively charged ions are pulled across the membrane toward a positively charged electrode in a water-based solution on the membrane’s wet side. The liquid solution dissolves the bicarbonate back into carbon dioxide, so it can be released and harnessed for fuel or other uses.
The electrical charge is used to speed up the transfer of bicarbonate across the membrane.
When they tested the system, which is small enough to fit in a backpack, the UIC scientists found that it had a very high flux — a rate of carbon capture compared with the surface area required for the reactions — of 3.3 millimoles per hour per 4 square centimeters. This is more than 100 times better than other systems, even though only a moderate amount of electricity (0.4 KJ/hour) was needed to power the reaction, less than the amount of energy needed for a 1 watt LED lightbulb. They calculated the cost at $145 per ton of carbon dioxide, which is in line with recommendations from the Department of Energy that cost should not exceed around $200 per ton.
[…]Polkit, previously known as PolicyKit, is a tool for setting up policies governing how unprivileged processes interact with privileged ones. The vulnerability resides within polkit’s pkexec, a SUID-root program that’s installed by default on all major Linux distributions. Designated CVE-2021-4034, the vulnerability has been given a CVSS score of 7.8.
Bharat Jogi, director of vulnerability and threat research at Qualys, explained in a blog post that the pkexec flaw opens the door to root privileges for an attacker. Qualys researchers, he said, have demonstrated exploitation on default installations of Ubuntu, Debian, Fedora, and CentOS, and other Linux distributions are presumed to be vulnerable as well.
“This vulnerability has been hiding in plain sight for 12+ years and affects all versions of pkexec since its first version in May 2009,” said Jogi, pointing to commit c8c3d83, which added a pkexec command.
The problem occurs when pkexec‘s main() function processes command-line arguments and argc – the ARGument Count – is zero. The function tries to access the list of arguments anyway, and ends up trying to use an empty argv – the ARGument Vector of command-line argument strings. As a result, out-of-bounds memory gets read and written, which an attacker can exploit to inject an environment variable that can cause arbitrary code to be loaded from storage and run by the program as root.
[…]
At least the exploitation technique proposed by Qualys – injecting the GCONV_PATH variable into pkexec‘s environment to execute a shared library as root – leaves traces in log files.
A funny thing happened on Google Drive overnight. Seemingly innocuous files started being flagged as violating the search behemoth’s terms of service over copyright infringement.
Dr Emily Dolson, assistant professor at Michigan State University, was one of those affected after she attempted to upload a file containing a single digit, “1”.
There wasn’t a lot of detail in the warning, only that Googles Drive’s Copyright Infringement policy had been violated and that no review could be requested for the restriction, both of which are a bit worrying for people concerned about the dead hand of AI being used as arbiter in such matters.
What had upset Google? The digit or the output04.txt filename? Certainly the number “1” does turn up in all manner of copyrighted works, although we don’t think anyone’s tried to trademark the character. Most recently, Snap made a spectacle of itself by trying to trademark the word “Spectacles”.
Could Google be trying to up the ante, and is it aware that Microsoft has its own cloud storage named OneDrive? Redmond already had to ditch SkyDrive after a well-known broadcaster took exception to it. We can’t imagine Nadella and co liking the sound of “Number Two Drive” for a variety of reasons.
More likely, the issue was more of a screw-up than conspiracy with both Google staffers and the Google Drive social media mouthpiece responding to confirm that the team was aware of the issue and working on it.
Because there’s always an Onion article where automation drives swathes of the IT world beyond satire.
Things seem OK now (at least as far as our testing is concerned), although we have asked Google to explain itself. We will update this piece if it does so.
Whatever the fix was, we suspect it wasn’t this. ®
A flying car capable of hitting speeds over 100mph (160kmh) and altitudes above 8,000ft (2,500m) has been issued with a certificate of airworthiness by the Slovak Transport Authority.
The hybrid car-aircraft, AirCar, is equipped with a BMW engine and runs on regular petrol-pump fuel.
It takes two minutes and 15 seconds to transform from car into aircraft.
The certification followed 70 hours of flight testing and more than 200 take-offs and landings, the company said.
[…]To be clear, data collection is far from an Amazon-specific problem; it’s pretty much par for the course when it comes to tech companies. Even Apple, a company vocal about user privacy, has faced criticism in the past for recording Siri interactions and sharing them with third-party contractors.
The issue with Amazon, however, is the extent to which they collect and archive your data. Just about everything you do on, with, and around an Amazon product or service is logged and recorded. Sure, you might not be surprised to learn that when you visit Amazon’s website, the company logs your browsing history and shopping data. But it goes far beyond that. Since Amazon owns Whole Foods, it also saves your shopping history there. When you watch video content through its platforms, it records all of that information, too.
Things get even creepier with other Amazon products. If you read books on a Kindle, Amazon records your reading activity, including the speed of your page turns (I wonder if Bezos prefers a slow or fast page flip); if you peered into your Amazon data, you might find something similar to what a Reuter’s reporter found: On Aug. 8 2020, someone on that account read The Mitchell Sisters: A Complete Romance Series from 4:52 p.m. through 7:36 p.m., completing 428 pages. (Nice sprint.)
If you have one of Amazon’s smart speakers, you’re on the record with everything you’ve ever uttered to the device: When you ask Alexa a question or give it a command, Amazon saves the audio files for the entire interaction. If you know how to access you data, you can listen to every one of those audio files, and relive moments you may or may not have realized were recorded.
Another Reuters reporter found Amazon saved over 90,000 recordings over a three-and-a-half-year period, which included the reporter’s children asking Alexa questions, recordings of those same children apologizing to their parents, and, in some cases, extended conversations that were outside the scope of a reasonable Alexa query.
Unfortunately, while you can access this data, Amazon doesn’t make it possible to delete much of it. You can tweak your privacy settings you stop your devices from recording quite as much information. However, once logged, the main strategy to delete it is to delete the entire account it is associated with. But even if you can’t delete the data while sticking with your account, you do have a right to see what data Amazon has on you, and it’s simple to request.
How to download all of your Amazon data
To start, , or go to Amazon’s Help page. You’ll find the link under Security and Privacy > More in Security & Privacy > Privacy > How Do I Request My Data? Once there, click the “Request My Data” link.
From the dropdown menu, choose the data you want from Amazon. If you want everything, choose “Request All Your Data.” Hit “Submit Request,” then click the validation link in your email. That’s it. Amazon makes it easy to see what the have on you, probably because they know you can’t do anything about it.
A new type of malware takes a decidedly more stealthy and hard-to-remove path into your OS — it hides in your BIOS chip and thus remains even after you reinstall your OS or format your hard drive.
Kaspersky has observed the growth of Unified Extensible Firmware Interface (UEFI) firmware malware threats since 2019, with most storing malware on the EFI System Partition of the PC’s storage device. However, a sinister development has been spotted over the New Year with a new UEFI malware, detected by Kasperksy’s firmware scanner logs, that implants malicious code into the motherboard’s Serial Peripheral Interface (SPI) Flash. The security researchers have dubbed this flash-resident UEFI malware ‘MoonBounce’.
[,…]
Below, a flow chart breaks down how MoonBounce boots and deploys from the moment your UEFI PC is switched on, through Windows loading, and into being a usable but infected PC.
(Image credit: Kaspersky Labs)
APT41 Fingerprints Detected
Another important branch of the work done by security researchers like Kaspersky is looking into who is behind the malware that it discovers, what the purposes of the malware are, and what specific targets the malware is primed for.
Concerning MoonBounce, Kaspersky seems pretty certain that this malware is the product of APT41, “a threat actor that’s been widely reported to be Chinese-speaking.” In this case, the smoking gun is a “unique certificate” that the FBI has previously reported as signaling the use of APT41-owned infrastructure. APT41 has a history of supply chain attacks, so this is a continuation of a central thread of APT41’s nefarious operations.
Safety Measures
To help avoid falling victim to MoonBounce or similar UEFI malware, Kaspersky suggests a number of measures. It recommends users keep their UEFI firmware updated directly from the manufacturer, verify that BootGuard is enabled where available, and enable Trust Platform Modules. Last but not least, it recommends a security solution that scans system firmware for issues so measures can be taken when UEFI malware is detected.
Turn your pictures into 3D stl files of lamp lithophanes, flat lithophanes, night light lithophanes, and more by using the lithophane makers below. Learn more about how to use LithophaneMaker.com by watching my YouTube tutorials. Click on a lithophane picture or title to go to the tool that created that lithophane. Instructions on how to use the lithophane makers are on their page, and general instructions on how to 3D print a lithophane are on the 3D Printing page. Give me feedback by joining the Lithophane Maker User’s group on Facebook.
Heart Lithophane Maker
Turn your pictures into beautiful heartfelt gifts for your loved ones! The new user interface for this tool lets you crop your pictures on the first page, then click the button at the top that says CLICK HERE TO VIEW LITHOPHANE for you to see the lithophane and adjust its dimensions. You can lower the value by the rendering resolution to make the lithophane look more like the final product, or increase the value to make the lithophane render quicker.
Lithophane Lamp Maker
Turn up to four pictures into a lithophane lamp model using this tool. The tool provides an interface that will work will most lamps. A cutout cylinder with a ledge makes it possible to put the lamp lithophane directly over the lamp’s light socket and underneath the light. The default settings work for a lamp that I have at my own house, but I suggest you measure the light bulb socket that you’re going to put the lithophane lamp over.
Lithophane Light Box Maker
Turn your photos into a lithophane light box. The lithophane light box was designed to easily take light sockets like the ones you can find here. You can design a customized lithophane light box and crop your photos in just a few minutes using this tool.
Night Light Lithophane Maker
Click the picture above to access the night light lithophane maker. The default settings for the night light lithophane make the lithophane with night lights can be bought here. This webtool gives you the ability to design the night light lithophane to be able to interface with almost any night light!
Flat Lithophane Maker
Turn a photo into a hangable flat lithophane stl with this tool. This tool automatically surrounds the lithophane with a frame and some holes for hanging the lithophane. Some twine and suction cups can be used to attach the lithophane to a window, and pretty much any will work. We used this twine and these suction cups.
Lithophane Globe Maker
Design a spherical lithophane with an optional lunar background. The lithophane interfaces with a light bulb through a cylindrical base, and can have a hole at the other end if desired. You can select the aspect ratio of your picture and crop it in this tool as well.
Curved Lithophane Maker
This lithophane design tool creates curved lithophanes or completely round votive lithophanes. You can adjust the dimensions of the lithophane that are shown in the picture to get exactly what you want.
Ceiling Fan Lithophane Maker
This image to stl generator turns pictures into a ceiling fan lithophane. You can turn up to four pictures into a cylindrical lithophane that has hooks that fit into a circular lithophane that is also designed here. The circular lithophane has 1 or 2 holes that allow you to attach to the ceiling fan’s pull string fixture.
Circular Lithophane Maker
This tool with crop an image into a circle and create a flat 3d stl from your photo. The 3d model can have a positive or negative image, so that you can make a lithophane or inverse with this tool. The 3d model is designed to be printed horizontally, and the model comes with a hole for attaching it to a string, hook, collar, or whatever you have in mind!
Color Lithophane Maker
This lithophane tool turns a picture into a the stl files you need to print a color lithophane.
Christmas Tree Lithophane Maker
Turn your picture into a Christmas Tree Lithophane with this tool! These lithophanes can be placed on a table, or hung from a tree. I have found compared two lighting options. This tea light is bright enough to illuminate the lithophane in regular room lighting, but has a battery life of 30 hours and I recommend a clamp diameter of 28.5mm for it. This tea light lasts for 100 hours, but doesn’t illuminate the lithophane well in a dark room (but not a bright one), and needs a clamp diameter of 36mm.
You might not want to play a Dark Souls game online for a while — not that you necessarily can. As Dexerto and The Verge report, attackers have discovered a security exploit in Dark Souls 3 (and potentially Elden Ring) for Windows that lets attackers remotely execute control and effectively hijack your PC. Streamers like The_Grim_Sleeper have learned about the potential damage first-hand — in his case, the intruder launched Microsoft PowerShell and ran a text-to-speech script blasting him for his gameplay.
The exploiter might not have malicious intent. A post on the SpeedSouls Discord claimed the hacker was trying to warn developer FromSoftware about the Dark Souls 3 vulnerability, but turned to compromising streamers to highlight the problem. Few people beyond the perpetrator are aware of how to use it, but there’s already a patch for the unofficial Blue Sentinel anti-cheat tool.
FromSoftware and its publisher Bandai Namco have since responded to the exploit. They’ve temporarily shut down the player-versus-player servers for Dark Souls 3 and its predecessors while the security team investigates the flaws.
To tackle climate change we need good data. This data exists; it is published by the International Energy Agency (IEA). But despite being an institution that is largely publicly funded, most IEA data is locked behind paywalls.
[…]
In 2020 we launched a campaign to unlock this data; we started on Twitter (one example), last year we wrote a detailed article about the problem here on OWID, and our letter in Nature.
[…]
The IEA has just announced that it aims to make all of its data and analysis freely available and open-access. This was put forward by the IEA’s executive director, Fatih Birol, and has been approved by its governing board already.
There is one step left. Next month – on February 2nd and 3rd – the IEA will ask for approval from its member countries. That means it is on the governments of the world’s rich countries to make this happen. If they do not approve it, it would be a missed opportunity to accelerate our action on addressing climate change.
This would be a massive achievement. The benefits of closing the small funding gap that remains greatly outweigh the costs.
There is now large support for the IEA data to be freely available – from researchers to journalists; policymakers to innovators. Many have called for the IEA data to be public. Many thanks to everyone who has joined in pushing this forwards – below we share the links to several articles, petitions, and open letters that have made this possible.
→ Open petition letter: Free IEA Data – A site by Skander Garroum and Christoph Proeschel on which you can write a letter to your country’s government.
