The New Tories - Ruthlessly Incompetent. Post Examples of Tory Stupidity Here
- Thread starter Kaka Tim
- Start date
DotCommunist
So many particulars. So many questions.
apparently gove was doing 'fireside chats' like he's FDR or something, bunch of cunts and their shit rally.
Artaxerxes
Look out, he's got a gnu!
IT is fickle and even big Devs frequently have trouble on launch days for online only games.
existentialist
Tired and unemotional
"If it can go wrong, it will" was never so true of both IT and public demonstrations of it.
When I used to do presentations of new software tools, I always made sure there were some gags and burble in there to cover the inevitable yet unexpected something-going-wrong scenario.
You learn a new respect for probability in that line...
When I used to do presentations of new software tools, I always made sure there were some gags and burble in there to cover the inevitable yet unexpected something-going-wrong scenario.
You learn a new respect for probability in that line...
Artaxerxes
Look out, he's got a gnu!
Commons Twitter account banned from tweeting vote results | House of Commons | The Guardian
Tory MPs argued a tweet on a trade bill amendment in July broke impartiality rules
MrSki
Who am I to say you're wrong
Hanbury Strategy: They’re getting rich from COVID-19
From the Good Law Project website. Hanbury Strategy: They’re getting rich from COVID-19 - Good Law Project
From the Good Law Project website. Hanbury Strategy: They’re getting rich from COVID-19 - Good Law Project
existentialist
Tired and unemotional
It's so easy to write something off as a "computer glitch", and yes - sometimes completely unforeseen "glitches" do happen that are beyond the control of anyone. But this isn't that - this is a design decision that should simply not have been made. The minute you add a complex layer - like a spreadsheet - into a system, you're introducing a level of potential cockup that should at least be foreseeable. So you either surround it with suitable checks and balances, or you find a simpler way of doing it. And then surround that with checks and balances. Even if it's just a bloody count, eg, "lines in file = x, records imported = y, O NOES, x != y"
This is not the computer's fault.
This is not the computer's fault.
PEBKAC
existentialist
Tired and unemotional
And the reason they didn't is pretty much the reason anyone who uses a spreadsheet instead of a database does so - because spreadsheets LOOK easy, and LOOK - to anyone who hasn't a fucking clue what's actually going on - as if they're doing a "database thing".They should've used a database, penny pinching whilst pillaging the public purse.
Which is fine - why should we expect yer average user to know that difference, any more than we'd expect the average person going to the doctor to distinguish between a stress headache and a brain haemorrhage.
But then that's why we have doctors. And IT specialists.
So the question I find myself asking is "how the hell did this harebrained idea get past an IT specialist?". I mean, it's not like Eric from Accounts cocking a snook at IT by deciding to set up a private little spreadsheet to scratch that itch IT won't do anything about...this is supposedly an integral part of a critical, large-scale system.
The mind boggles. And, of course, we'll never get any answers. Even if they don't just ignore it, they'll almost certainly be claiming "commercial confidentiality" or something.
Cunts.
SpookyFrank
A cheap source of teeth for aquarium gravel
SpookyFrank
A cheap source of teeth for aquarium gravel
I'm not exactly Bill Gates but even I know that.
teqniq
DisMembered
I did not, but then again I'm not overly familiar with the office suite.
existentialist
Tired and unemotional
I'm a former IT specialist...and I didn't know what the maximum size of an Excel spreadsheet is before all this. But I do know that it's finite.
In, to be fair, exactly the same way that I know that the disk space on which a database is stored is finite...so one of the, ahem, fairly important roles of a system designer is to look at scale, and ensure that whatever I'm using has sufficient capacity. And not just x + 10% "sufficient capacity", more like x * 4. And then you make sure you've got monitoring and checks, so that when your previously nearly-unlimited expanse of disk space (or columns) is down to half what it was, you have fair warning, and can do something about it (which is easier with disk space than it is with a fucking spreadsheet).
I guess it comes down to the sort of people who think they're "programming" when they set a website up on Wix
In, to be fair, exactly the same way that I know that the disk space on which a database is stored is finite...so one of the, ahem, fairly important roles of a system designer is to look at scale, and ensure that whatever I'm using has sufficient capacity. And not just x + 10% "sufficient capacity", more like x * 4. And then you make sure you've got monitoring and checks, so that when your previously nearly-unlimited expanse of disk space (or columns) is down to half what it was, you have fair warning, and can do something about it (which is easier with disk space than it is with a fucking spreadsheet).
I guess it comes down to the sort of people who think they're "programming" when they set a website up on Wix
stdP
One Swiss dinosaur in Polly Gosling's anorak hood
They should've used a database, penny pinching whilst pillaging the public purse.
As someone who's written similar things... isn't it actually easier to get data out of a web site in to a DB than it is in to an excel spreadsheet?! The interaction between "language used by the web application" and "connecting to the database" is one of the first lessons you ever learn about this sort of thing because it's the most common use-case ever... to get it into excel (.xls mind, not the new-fangled .xlsx) you typically have to jump through all sorts of fiery hoops (which are usually carefully watched by people like me flinging "who came up with this stupid design?" barbs around). Cummings and co throw around so much Bullshot Moonshit 2.0 that I expected the error to at least be a trendy one, their JSON overclocking the blockchain django or something similar.
That said, it's another dog-shit fiasco overseen by Dido and her outsourced sense of responsibility so why am I really surprised. Oh well, at least it's not for anything important!
The "old" excel format (referred to above as .XLS going by the file extension since I'm not sure what it's called properly) had a limit of ~64k rows (aka 65536 or 2^16), the "new" excel format has a limit of a million or so IIRC. Either of them is a stupid format for holding this sort of data.
Edit: BBC explainer on the issue. Apparently they were using CSVs as an intermediate format (no limit on those as they're just text files) but when opening them with excel it silently truncated them. Quoth the article "one expert suggested that even a high-school computing student would know that better alternatives exist".
Excel: Why using Microsoft's tool caused Covid-19 results to be lost
The decision to use a spreadsheet format that dates back to the 1980s has proved to be unwise.
www.bbc.co.uk
Last edited:
Artaxerxes
Look out, he's got a gnu!
It's been bad enough supporting a dozen companies whose accounts departments (and others) have 12 year old excel documents over 100mb so fuck knows whose bright idea it was to whack this lot into a spreadsheet.
two sheds
Least noticed poster 2007
BBC saying it was PHE's fault
Excel: Why using Microsoft's tool caused Covid-19 results to be lost
The decision to use a spreadsheet format that dates back to the 1980s has proved to be unwise.
www.bbc.co.uk
existentialist
Tired and unemotional
TBH, if you own the systems which the website data is based on, then people don't need to scrape data off a website. You can set up all kinds of ways of moving structured data from server to client, in ways that permit of web display or incorporating into some kind of local datastore - off the top of my head, a decent REST API and a data interchange format like JSON would be a still pretty low-tech way of achieving the job, but without all the nightmare shit of sticking it in a spreadsheet, etc.As someone who's written similar things... isn't it actually easier to get data out of a web site in to a DB than it is in to an excel spreadsheet?! The interaction between "language used by the web application" and "connecting to the database" is one of the first lessons you ever learn about this sort of thing because it's the most common use-case ever... to get it into excel (.xls mind, not the new-fangled .xlsx) you typically have to jump through all sorts of fiery hoops (which are usually carefully watched by people like me flinging "who came up with this stupid design?" barbs around). Cummings and co throw around so much Bullshot Moonshit 2.0 that I expected the error to at least be a trendy one, their JSON overclocking the blockchain django or something similar.
That said, it's another dog-shit fiasco overseen by Dido and her outsourced sense of responsibility so why am I really surprised. Oh well, at least it's not for anything important!
The "old" excel format (referred to above as .XLS going by the file extension since I'm not sure what it's called properly) had a limit of ~64k rows (aka 65536 or 2^16), the "new" excel format has a limit of a million or so IIRC. Either of them is a stupid format for holding this sort of data.
Edit: BBC explainer on the issue. Apparently they were using CSVs as an intermediate format (no limit on those as they're just text files) but when opening them with excel it silently truncated them. Quoth the article "one expert suggested that even a high-school computing student would know that better alternatives exist".
Excel: Why using Microsoft's tool caused Covid-19 results to be lost
The decision to use a spreadsheet format that dates back to the 1980s has proved to be unwise.www.bbc.co.uk
My take on this is that they've done the classic "urgent" thing, and focused on lashing together disparate systems, probably on the basis of local domains of knowledge ("Hi, I'm TED - I know Excel backwards, and I reckon we could do a nice job on data reformatting with a few macros and formulas") rather than someone taking a more...architectural view.
But that - my own knowledge domain - makes me realise that this is how ALL of the Government's decisions seem to be taken. It's all about bodging it together in a hurry - laws, computer systems, social measures - and not feeling under any obligation to take any responsibility for the consequences of failure. It's just the same as the infamous ATOS/DWP alliance, providing plausible deniability for both, and accountability for none.
But then that's not incompetence, so much as establishing the framework in which incompetence can flourish.
Artaxerxes
Look out, he's got a gnu!
238 million quid for interview classes and CV writing for those out of work due to Covid.
www.theguardian.com
New jobs coaches will help people back to work, says Rishi Sunak
Jets scheme will help those laid off during pandemic, chancellor to tell Tory conference
existentialist
Tired and unemotional
Fucksake.238 million quid for interview classes and CV writing for those out of work due to Covid.
New jobs coaches will help people back to work, says Rishi Sunak
Jets scheme will help those laid off during pandemic, chancellor to tell Tory conference
stdP
One Swiss dinosaur in Polly Gosling's anorak hood
But that - my own knowledge domain - makes me realise that this is how ALL of the Government's decisions seem to be taken. It's all about bodging it together in a hurry - laws, computer systems, social measures - and not feeling under any obligation to take any responsibility for the consequences of failure. It's just the same as the infamous ATOS/DWP alliance, providing plausible deniability for both, and accountability for none.
But then that's not incompetence, so much as establishing the framework in which incompetence can flourish.
As far as I've been able to ascertain, the web server(s) were dumping data out to CSV (which is in itself a pretty terrible interchange format for anything remotely complicated - much better to use JSON or XML) and then consolidating CSVs in to a single spreadsheet, instead of consolidating it in a database and using a web or some other frontend application to process individual cases. If you try and add too many rows (or columns for that matter) in to a ye olde Excel, it'll silently drop them which is why anyone who understands data integrity breaks out in a cold sweat whenever anyone mentions spreadsheets. Excel is, at best, a presentation format and should never be used as a data processing format for reasons that are too long to list. There's a million better ways to have done this, none of which involve excel, and I don't entirely understand how things got to this point (but then I get the same apoplexy about Horizon as well).
Worse still, this means there's likely several (almost certainly unencrypted) copies of this file floating about containing the data of every single identified case. Another big reason for using a proper database with a frontend application in that it makes it relatively easy to implement access controls, something you can't do with raw files holding all the data.
I don't disagree with your assessment that so many of these IT decisions are bodged together in the worst way possible, but I struggle to believe there isn't a single person there who understands ETL and other data warehousing concepts, as well as not being aware of limitations in excel that have been known for 20 years. If this had happened back in march or april I might have believed it as an interim solution... but six months later and is breaks and no-one seemed be aware of it? I guess maybe I don't see the difference between establishing the framework in which incompetence can flourish and incompetence itself (unless you design-in the structural incompetence as a form of malice).
Anyway... I'm derailing in to spittle-flecked geek outrage. Suffice to say I think this is a perfect example of ruthless incompetence and I'm looking forward to blame being shifted so Dido doesn't look any worse than she does already.
At the rate the hospitality industry at least is going, do we have a sweepstake on how many jobs there'll be to go around in six months' time? Hopefully Rishi will also finance positive thinking seminars and spiritual wellness fora at the soup kitchens. Oh look, several positions have opened up for al-fresco seating redeployment facilitators for the new flagship of the White Star line!
Last edited:
Similar threads
