Search the KHIT Blog

Showing posts sorted by relevance for query natural language processing. Sort by date Show all posts
Showing posts sorted by relevance for query natural language processing. Sort by date Show all posts

Tuesday, May 23, 2017

"Assuming / Despite / If / Then / Therefore / Else..." Could AI do "argument analysis?"


When I was a kid in grade school, back prior to indoor plumbing, it was just broadly referred to as "reading comprehension" -- "What was the author's main point?" Did she provide good evidence for her point of view? Do you agree or disagree with the author's conclusion? Why? Explain..."

The oral equivalent was taught in "debate teams" prep. #NLP

Now along comes the part of "AI" technology R&D (Artificial Intelligence) known by its top-level acronym "NLP" (Natural Language Processing). We see increasing discourse on developments in "Machine Learning," "Deep Learning," "Natural Language Generation" (NLG) and "Natural Language Understanding" (NLU).
There's been a good bit of chatter of late in the Health IT news about the asserted utility of NLP. See here as well.
I am interested in particular in the latter (NLU), most specifically as it pertains to rational "argumentative" discourse (mostly of the written type). e.g., "Critical Thinking" comes to mind (I was lucky to get to teach it for a number of years as an adjunct faculty member). I was subsequently accorded the opportunity to teach a graduate seminar in the higher-level "Argument Analysis."

From my grad seminar syllabus:
We focus on effective analysis and evaluation of arguments in ordinary language. The "analysis" part involves the process of getting at what is truly being argued by a proponent of a position on an issue. Only once we have done that can we begin to accurately assess the relative merits of a proposition—the "evaluation" phase. These skills are essential to grasp if we are to become honest and constructive contributors to debate and the resolution of issues.

Our 24/7 global communications civilization is awash in arguments ranging from the trivial to grand themes of moral import. Advocates of every stripe and theme pepper us relentlessly with persuasion messages ranging from the "short and sweet" to the dense and inscrutable. We have more to consider and evaluate than time permits, so we must prioritize. This we often do by making precipitous snap judgments—"Ready-Shoot-Aim"—which then frequently calcify into prejudice. The sophistication and nuance of language enables a savvy partisan to entice us into buying into an argument perhaps not well supported by the facts and logic...
I first encountered "Argument Analysis" in the fall of 1994 as an "Ethics & Policy Studies" graduate student myself. I chose for my first semester paper an analytic deconstruction of the PNHP 1994 JAMA paper "A Better-Quality Alternative: Single-Payer National Health System Reform."

The first two opening paragraphs:
MANY MISCONSTRUE US health system reform options by presuming that "trade-offs" are needed to counter-balance the competing goals of increasing access, containing costs, and preserving quality. Standing as an apparent paradox to this zero-sum equation are countries such as Canada that ensure access to all at a cost 40% per capita less, with satisfaction and outcomes as good as or better than those in the United States. While the efficiencies of a single-payer universal program are widely acknowledged to facilitate simultaneous cost control and universal access, lingering concerns about quality have blunted support for this approach.
Quality is of paramount importance to Americans. Opponents of reform appeal to fears of diminished quality, warning of waiting lists, rationing, and "government control." Missing from more narrow discussions of the accuracy of such charges is a broader exploration of the quality implications of a universal health care program. Conversely, advocates of national health insurance have failed to emphasize quality issues as key criteria for reform, often assuming that we have "the best medical services in the world." They portray reform primarily as extending the benefits of private insurance to those currently uninsured, with safeguards added to preserve quality.
For the "analysis" phase I undertook to examine and "flowchart" the subordinate arguments' logic of the 49 paragraphs of assertions comprising the PHNP article, numbering every argument statement as "paragraph(n), sentence(n.n), and sub-sentence truth-claim clause(n.n.a,b,c...) where warranted" as evident by close reading of the text. My full (pdf) copy of the paper is parked here.

Click to enlarge.
Dotted lines denote a "despite" (a.k.a. "notwithstanding") statement, whereas solid lines depict "because-therefore" premise-to-conclusion movement in the direction of the arrowheads.

It was tedious. The bulk of the first 25 pages of the 56 page paper comprised this analytic "flowcharting" visualization helpful for what the late Steve Covey would characterize as a crucial "seek first to understand" effort. The remaining 31 pages subsequently focused on my (in large measure subjective) critical evaluation of the logic and evidence provided by the authors.
BTW: I'm certain I didn't get everything exactly right on the "analysis" side (or the eval side, for that matter). It was my first run at this type of thing. And, I had a second course to deal with at the time ("History of Ethics," 11 required texts) and was still working full-time at my Medicare QIO job.
Look at sentence 1.1, for example. You could nit-pick my decision, by splitting it up into "b" and "a" because-therefore clauses. Because "presuming trade-offs are needed," therefore "Many misconstrue..." Not that it'd have made a material difference in the analysis, but, still.
UPDATE: per the topic of my 1994 paper, Dr. Danielle Ofri in the news:
Americans Have Realized They Deserve Health Care
How long until they accept that the only way to guarantee it is through single-payer?
I have a good 100 hours or so in that one grad school paper. Imagine trying to do that to an entire book. Utterly impractical. So, we mostly suffer our "confirmation bias" and similar heuristic afflictions and jump to premature conclusions -- the bells we can't un-ring.

Hmmm...


Could we develop an AI NLU "app" for that? (I don't underestimate the difficulty, given the myriad fluid nuances of natural language. But, still...)
Thinking about NLP applicability to digital health infotech (EHRs), the differential dx SOAP method (Subjective, Objective, Assessment, and Plan) is basically an argument process, no? You assemble and evaluate salient clinical evidence (the "S" and the "O" data, whether numerical, encoded, or lexical narrative), which point in the aggregate to a dx conclusion and tx decision (the "A" and the "P"). I guess we'll see going forward whether applied NLP has any material net additional utility in the dx arena, or whether it will be just another HIT R&D sandbox fad.
Logic visualization software is not exactly news. In the late 80's I developed an instrumentation statistical process control program for the radiation lab where I worked in Oak Ridge -- the "IQCstats" system (pdf). Below is one page of the 100 or so comprising the logic flowcharts set included in my old 2" bound "User and Technical Guide" manual.

Click to enlarge

The flowcharts were generated by an "app" known as "CLEAR," which parsed my source code logic and rendered a complete set of flowcharts.

While "critical evaluation" of arguments proffered in ordinary language might not lend itself to automated digital assessment (human "judgments"), mapping the "Assuming / Despite / If / Then / Therefore / Else" logic might indeed be do-able in light of advances in "Computational Linguistics" (abetted by our exponentially increasing availability of ever-cheaper raw computing power).

Below, my graphical analogy for the fundamental unit of "argument" (a.k.a. "truth claim").

Click to enlarge



Any complex argument arises from assemblages of the foregoing "atomic" and "molecular" "particles" (once you've weeded through and discarded all of the "noise").
I should add that most of what I'm interested in here goes to "informal/propositional logic" in ordinary language. Formal syllogistic logic (e.g., formal deductive "proofs") comprise a far smaller subset of what we humans do in day-to-day reasoning.
English language discourse, recall, beyond the smaller "parts of speech," is comprised of four sentence types:
  1. Declarative;
  2. Interrogative;
  3. Imperative;
  4. Exclamatory.
We are principally interested in the subset of declaratives known as "truth claims" -- claims in need of evaluation prior to acceptance -- though we also have to be alert to the phony "interrogative" known as the "loaded question," i.e., an argument insinuation disingenuously posed as a "have-you-stopped-beating-your-wife" type of query. (Then there's also stuff like subtle inferences, ambiguities, and sarcasm, etc that might elude AI/NLU.)

NLP AND LINGUISTICS

It occurs to me that, notwithstanding my longstanding chops on the verbal/written side, I've never had any formal study in "linguistics," much less its application in NLP. Time to start reading up.

Introduction
Natural languages are the languages which have naturally evolved and used by human beings for communication purposes, For example Hindi, English, French, German are natural languages.  Natural language processing or NLP (also called computational linguistics) is the scientific study of languages from computational perspective. natural language processing (NLP) is a field of computer science and linguistics concerned with the interactions between computers and human (natural) languages. Natural language generation systems convert information from computer databases into readable human language. Natural language understanding systems convert samples of human language into more formal representations such as parse trees or first order logic that are easier for computer programs to manipulate. Many problems within NLP apply to both generating and understanding; for example, the computer must  be able to model morphology (the structure of words) in order to understand an English sentence, and a model of morphology is also needed for producing a grammatically correct English sentence, i.e., natural language generator.

NLP has significant overlap with the field of computational linguistics, and is often considered a subfield of artificial intelligence. The term natural language is used to distinguish human languages (such as Spanish, Swahili, or Swedish) from formal or computer languages (such as C++, Java, or LISP).  Although NLP may end comp us both  text and speech, work on speech processing is conventionally done in a separate field.

In NLP, the techniques are developed which aim the computer to understand the commands given in natural language and perform according to it. At present, to work with computer, the input is required to be given in formal languages. The formal languages are those languages which are specifically developed to communicate  with computer and are understood by machine, e.g., FORTRAN, Pascal, etc. Obviously, to communicate with computer, the study of these formal languages is required. Understanding these languages is  cumbersome and requires additional efforts to understand these. Hence, it limits their applications in computer. As compared to this, the communication in natural language will facilitate the functioning and communication with computer easily and in user-friendly way.

 Natural language processing is a significant area of artificial intelligence because a computer would be considered intelligent  if it can understand the commands given in natural language instead of C, FORTRAN, or Pascal. Hence, with the ability of computers to understand natural language it becomes much easier to communicate with computers. Also the natural language processing can be applied as a productivity tool in applications ranging from summarization of news to translate from one language to another. Though, the surface level processing of natural languages seems to be easy the deep level processing of natural languages, understanding of implicit messages and intentions of the speaker are extremely difficult avenues...
Ya have to wonder whether that was written by a computer. Minimally, a non-native English speaker/writer.

I've also just read up on "linguistics" broadly via a couple of short books, just to survey the domain.


The real meat comes here:


801 pages of dense, comprehensive detail.
Introduction
The field of computational linguistics (CL), together with its engineering domain of natural language processing (NLP), has exploded in recent years. It has developed rapidly from a relatively obscure adjunct of both AI and formal linguistics into a thriving scientific discipline. It has also become an important area of industrial development. The focus of research in CL and NLP has shifted over the past three decades from the study of small prototypes and theoretical models to robust learning and processing systems applied to large corpora. This handbook is intended to provide an introduction to the main areas of CL and NLP, and an overview of current work in these areas. It is designed as a reference and source text for graduate students and researchers from computer science, linguistics, psychology, philosophy, and mathematics who are interested in this area.
The volume is divided into four main parts. Part I contains chapters on the formal foundations of the discipline. Part II introduces the current methods that are employed in CL and NLP, and it divides into three subsections. The first section describes several influential approaches to Machine Learning (ML) and their application to NLP tasks. The second section presents work in the annotation of corpora. The last section addresses the problem of evaluating the performance of NLP systems. Part III of the handbook takes up the use of CL and NLP procedures within particular linguistic domains. Finally, Part IV discusses several leading engineering tasks to which these procedures are applied...

(2013-04-24). The Handbook of Computational Linguistics and Natural Language Processing (Blackwell Handbooks in Linguistics) (p. 1). Wiley. Kindle Edition.
Interesting. BTW, nice summation of Computational Linguistics on the Wiki.
Computational linguistics is an interdisciplinary field concerned with the statistical or rule-based modeling of natural language from a computational perspective.

Traditionally, computational linguistics was performed by computer scientists who had specialized in the application of computers to the processing of a natural language. Today, computational linguists often work as members of interdisciplinary teams, which can include regular linguists, experts in the target language, and computer scientists. In general, computational linguistics draws upon the involvement of linguists, computer scientists, experts in artificial intelligence, mathematicians, logicians, philosophers, cognitive scientists, cognitive psychologists, psycholinguists, anthropologists and neuroscientists, among others.

Computational linguistics has theoretical and applied components. Theoretical computational linguistics focuses on issues in theoretical linguistics and cognitive science, and applied computational linguistics focuses on the practical outcome of modeling human language use...
"applied computational linguistics focuses on the practical outcome of modeling human language..."

Like, well, NLU Argument Analytics?

UPDATE: I'm hitting a motherload of good stuff in Chapter 15 of "The Handbook..." on "computational semantics."

After getting up to speed on the technical concepts and salient details, perhaps the next step would involve learning Python.

"This book offers a highly accessible introduction to natural language processing, the field that supports a variety of language technologies, from predictive text and email filtering to automatic summarization and translation. With it, you'll learn how to write Python programs that work with large collections of unstructured text. You'll access richly annotated datasets using a comprehensive range of linguistic data structures, and you'll understand the main algorithms for analyzing the content and structure of written communication.

Packed with examples and exercises, Natural Language Processing with Python will help you:
  • Extract information from unstructured text, either to guess the topic or identify "named entities"
  • Analyze linguistic structure in text, including parsing and semantic analysis
  • Access popular linguistic databases, including WordNet and treebanks
  • Integrate techniques drawn from fields as diverse as linguistics and artificial intelligence
This book will help you gain practical skills in natural language processing using the Python programming language and the Natural Language Toolkit (NLTK) open source library. If you're interested in developing web applications, analyzing multilingual news sources, or documenting endangered languages -- or if you're simply curious to have a programmer's perspective on how human language works -- you'll find Natural Language Processing with Python both fascinating and immensely useful."
apropos of this topic generally, a couple of prior posts of mine come to mind. See "The Great A.I. Awakening? Health Care Implications?" and "Are structured data now the enemy of health care quality?"

Tangentially, my post of July 2015 "AI vs IA: At the cutting edge of IT R&D" as well.

So, could we use digital NLU technology to passably analyze natural language arguments, rather than just turning lab data and ICD-10 codes into SOAP narratives (and the converse)?

Me and my crazy ideas. Never gonna make it into any episodes of "Silicon Valley" (NSFW).

Perhaps our Bootcamp Insta-Engineer pals at ZIPcode Wilmington could have a run at Argumentation NLU?

Seriously, how about a new subset of CL tech R&D, "NLAA" -- "Natural Language Argument Analysis?"
__

UPDATE: RECENT NLG REPORTAGE

From Wired:
What News-Writing Bots Mean for the Future of Journalism
Joe Keohane, 02.16.17

WHEN REPUBLICAN STEVE King beat back Democratic challenger Kim Weaver in the race for Iowa’s 4th congressional district seat in November, The Washington Post snapped into action, covering both the win and the wider electoral trend. “Republicans retained control of the House and lost only a handful of seats from their commanding majority,” the article read, “a stunning reversal of fortune after many GOP leaders feared double-digit losses.” The dispatch came with the clarity and verve for which Post reporters are known, with one key difference: It was generated by Heliograf, a bot that made its debut on the Post’s website last year and marked the most sophisticated use of artificial intelligence in journalism to date.

When Jeff Bezos bought the Post back in 2013, AI-powered journalism was in its infancy. A handful of companies with automated content-generating systems, like Narrative Science and Automated Insights, were capable of producing the bare-bones, data-heavy news items familiar to sports fans and stock analysts. But strategists at the Post saw the potential for an AI system that could generate explanatory, insightful articles. What’s more, they wanted a system that could foster “a seamless interaction” between human and machine, says Jeremy Gilbert, who joined the Post as director of strategic initiatives in 2014. “What we were interested in doing is looking at whether we can evolve stories over time,” he says...
More and more examples abound on the NLG side of things. Just Google "written by AI."

UPDATE: SPEAKING OF NEWS

I cited this excellent book a while back.


Re: Chapter 8, "Computational Journalism"
In 2009, Fred Turner and I wrote: “What is computational journalism? Ultimately, interactions among journalists, software developers, computer scientists and other scholars over the next few years will have to answer that question. For now though, we define computational journalism as the combination of algorithms, data, and knowledge from the social sciences to supplement the accountability function of journalism.”

Hamilton, James T. (2016-10-10). Democracy’s Detectives (Kindle Locations 10750-10753). Harvard University Press. Kindle Edition.
NLP seems an obvious fit, 'eh?
__

UPDATE

Check this out:

Study the intersection of language and technology and place yourself at the forefront of a dynamic field by earning a Master of Science in Computational Linguistics from the University of Washington.

The powerful connections between text, human speech and computer technology are having a growing impact on society and our everyday lives. In this program, you can explore a discipline that has applications in a wide variety of fields – including business, law and medicine – and incorporates such diverse technologies as predictive text messaging, search engines, speech recognition, machine translation and dialogue systems...
Nice. That's something I would do in a heartbeat. Here's their web link.


Seattle is a special place for me to begin with. Both of my daughters were born there. (I'm writing this while sitting with my younger daughter as she goes through round 2 of her chemo tx.)

Seattle. Those were the days...
(With much gratitude for the Statute of Limitations.)
I still have numerous rock-solid friendships there. Sadly, recently lost one. He succumbed after a terrible 11 battle with Mantle Cell Lymphoma. He was a younger brother to me. Without qualification one of the best drummers on the planet. He could've played for Sting.

CODA

From The Atlantic:
Rethinking Ethics Training in Silicon Valley
“If technology can mold us, and technologists are the ones who shape that technology, we should demand some level of ethics training for technologists.”

- Irina Raicu
I work at an ethics center in Silicon Valley.

I know, I know, “ethics” is not the first word that comes to mind when most people think of Silicon Valley or the tech industry. It’s probably not even in the top 10. But given the outsized role that tech companies now play, it’s time to focus on the ethical responsibilities of the technologists who help shape our lives...
Yeah, I know, the jokes just write themselves. "Silicon Valley" and "ethics" in the same sentence?

I was not even aware of this place.



I will have to study up on them and report further.

apropos, see my 2015 post "The old internet of data, the new internet of things, and 'Big Data' and the evolving internet of YOU."

UPDATE

See my follow-on post "Continuing with NLP: a $4,200 'Study'."

OCT 2021 UPDATE


Wow. Just finished this riveting book. No, AI/NLU will not be doing ordinary language Argument Analysis and Evaluation anytime soon, if ever. Read it to understand precisely why. A masterwork.
____________

Wednesday, August 23, 2017

On deck, the 2017 Health 2.0 Conference, Oct 1-4

"This October 2,000 of the best and brightest will gather to discuss, witness, and share the leading cutting-edge innovations transforming today’s global health care system. Health 2.0’s flagship event connects thought leaders, providers, innovators, investors, and start-ups for four days packed full of curated discussions, demos, exhibits and networking…"

Conference block rate for the onsite Hyatt Regency hotel expires Sept 7th. Mercifully, the NFL 49ers play Arizona at Arizona on October 1st, so there won't be that huge traffic mess around Levi Stadium.

I booked my hotel rez starting on Saturday September 30th so I can hit the ground running Sunday morning. Joe Flower will keynote the Provider Symposium Sunday morning.

BTW, Sept 30th is the final day of federal FY 2017. President Trump is threatening a government shutdown to try to coerce congressional funding of his Mexico Wall (yeah, right). More seriously, the "Freedom Caucus" right wing Tea Party faction of the GOP wants another federal shutdown to try to get their way on the federal debt ceiling. My note on this now-hardy perennial a couple of years ago on another of my blogs:
Let's be very clear here: While "defunding" and transiently shutting down the government by failing to pass budget legislation is a political act within the bounds of tripartite government (notwithstanding its inanity), intentionally defaulting on the public debt is a separate and explicit Constitutional violation of legislative branch members' Oaths of office -- notwithstanding the GOP extremists' conflating attempt to glom it all together. to wit,

Section 4 of the 14th Amendment declares that "the validity of the public debt shall not be questioned." Section 5 subsequently states that the Congress has the sole responsibility over this question.

More broadly, as set forth in Article I, Sections 7 and 8, only Congress has the authority to appropriate funds and levy taxes. They can appropriate funds by raising taxes and/or borrowing from the private credit markets in the name of the nation. If they choose the latter, they are required to see to it that payment is always honored, even should that mean raising taxes or cutting other program expenditures. Neither the President not the Supreme Court has any Constitutional authority here. Should honoring the public debt require a presidential veto override, so be it. Muster the requisite votes. Should it require taking out "poison pill" provisions in order to pass debt service legislation, so be it. Those are among the legitimate mechanics of governance, in this case devolving to the Congress and no other entity.
..
During the 2013 Health 2.0 Conference we had a number of federal officials in attendance (e.g., ONC, CMS) to speak. Because of the 2013 federal shutdown they had to leave early and return to DC.

We'll see.

HIMSS

Interestingly, this will be the first Health 2.0 Conference under HIMSS ownership (should we call it "The 1st Annual HIMSS 2.0 Conference?")


I find it a bit curious that there is no prominent, static mention/link (as of today) regarding the 2017 Health 2.0 Conference on the HIMSS homepage. What's up with that? (Mention of the Conference does appear transiently amid a rotating group of small horizontal banner ads at the top of the page. Hit "Ctrl-R" repeatedly to see if you can get it to pull up.)

I went to the HIMSS Facebook page.


OK. Nyet. Nada. Zilch.

They do cite this on the HIMSS homepage at the right side "top of the fold":


Interesting timing overlap. I just searched through the agenda. Maybe I've missed something, but I see no mention of "Health IT Week." (Neither do repeated spelling-variant searches on the health2con.com site turn up any mention.) Wonder if they'll have an exhibit hall booth presence?

Now that HIMSS owns the Health 2.0 Conferences, I hope I don't get any photography grief, like I encountered at the HIMSS16 closing Keynote.

BTW, loves my New Yorker...


NLP on the agenda?

Searching the conference agenda also fails to turn up any mention of the acronym "NLP" or its referent phrase "Natural Language Processing," on which I've previously ruminated here and here.

I continue to study and follow the topic. Ran across this the other day (which I've excerpted below):
FDA, UCSF-Stanford CERSI, and San Francisco State University Collaborative WorkshopJune 15, 2017

Goals and Objectives:

The objective of this workshop was to identify current and emerging natural language processing (NLP) efforts being applied to unstructured text such as clinical notes or narratives in electronic health records (EHRs). The workshop provided insights into utility and challenges in designing and implementing NLP systems to capture relevant or missing information from clinical notes or text for conducting postmarketing safety surveillance and informing the design and execution of clinical trials for medical products, which include drugs, biologics, and devices. The workshop included panel discussion sessions to provide stakeholders with a forum to discuss natural language processing with experts in the field.

The workshop focused on whether NLP can be applied to unstructured text in clinical notes to:

  • Identify indication or reason for medical product use, adverse outcomes or events associated with use of these products, and confounders or personal behaviors that may modify risks associated with use of these products
  • Support protocol design, feasibility, recruitment efforts and execution of clinical trials
Use of Natural Language Processing to Extract Information from Clinical Text: Summary of the FDA Workshop
 

A public workshop organized by the U.S. Food and Drug Administration (FDA), the UCSF-Stanford Center of Excellence in Regulatory Science and Innovation (UCSF-Stanford CERSI), and San Francisco State University was held at the FDA White Oak Campus on June 15, 2017. The objective of the workshop was to identify current and emerging natural language processing (NLP) efforts being applied to unstructured text such as clinical notes or narratives in electronic health records (EHRs)…

While this workshop focused on NLP, it was noted by several speakers that it is only one part of the solution pipeline (sequence of software tools) – it is most often preceded by complex data acquisition and pre-processing, and followed up with some combination of machine learning or rule-based systems to produce desired decisions or interpretations. In many cases, NLP is in fact performed as a set of rules, or has been replaced with text mining and statistical methods with good success in specific areas especially when a large number of records are available. 


Classical NLP technology, designed for regular text, encounters significant challenges in processing information from clinical text in medical records such as EHRs. The challenges come from two key areas: a) NLP was not originally designed to process data in this format and with the “noise” inherent in EHRs, and b) most NLP algorithms require large gold standard databases for training ground truth data, which are hard to come by. It was also noted that for NLP to be successful in these areas, it will need to adequately handle negative statements (negations) and medical context. 


In spite of the above challenges, a literature review presented by one of the speakers showed that NLP and related methods were successful for some specific applications (e.g., radiology diagnosis, filling in certain EHRs, extraction of specific medical terms, etc.). Since each of these applications were developed separately, the issue of generalization remains.
To move forward, several possible approaches and directions were identified by one or more speakers/panelists:
  • Given that NLP is only part of the analysis pipeline that may include (deep) machine learning, it is important to optimize the whole pipeline, from data capture to final data/decision representation.
  • Leveraging new analytics methods that reduce dependence on large training databases (e.g., deep learning, CNN, text mining etc.) would be beneficial.
  • Developing general solutions using a single generalizable NLP/analytics method is difficult, so it may be worthwhile to work on solving specific problems first, then analyzing commonalities among successful solutions, leading to possible generalizations…
DATA
Electronic Health Records (EHRs) and other structured clinical documents were originally designed for very different purposes (patient care, billing, reporting). The clinical notes or text contained in these documents may contain additional information and context about the medical encounter but the clinical information it contains is non-standardized, has errors, typos, omissions and often miss key information necessary for envisioned FDA applications (e.g., confounders, prescriber and patient intent or behavior). This poses significant challenges for classical NLP tools designed to work on regular text. Further complicating the situation, the applications of NLP addressed at the workshop are many, and often require information and context not originally coded or missing in EHRs and related documents (e.g., confounders, temporal components, state of instrumentations)…
Leveraging a variety of complementary data sources (including other patient records, observations during the exam etc.) in addition to EHRs would help to provide missing information, redundancy, as well as context, all of which are important for making the correct decisions…

TOOLS AND RESOURCES
The following ideas were suggested by one or more speakers/panelists as possibilities to help develop tools…


  • Using and leveraging best practices of open source software.
  • Developing software environments in the form of interactive workbench where one can easily create and test analytic pipelines (aggregate of software tools used) by integrating available tools, as has been done for other areas.
  • Provide and disseminate open source NLP and machine learning tools with adequate distribution, licensing and documentation.
Availability of accessible, easy to share gold standard databases remains critical. The positive experience of other areas on machine learning where such databases have helped to make significant advances needs to be leveraged. This remains, however, a technical/cost issue as well as a policy and legal issue due to data privacy considerations…
Recall from my earlier posts, there are two materially differing subtopics: [1] NLG, Natural Language Generation (expressing "structured" numerical data and informatics codes in narrative form -- relatively old news), and [2] NLU, Natural Language Understanding, the far more difficult area (and the focus of my abiding dubiety).

Hope I find some good NLP stuff at the Conference (amid my myriad other topics of KHIT interest).

UPDATE: NLP NEWS ITEM
Apple's Siri latest target in string of natural language patent lawsuits
One-man company Word to Info on Friday expanded a string of patent lawsuits over natural language processing technology — active cases involve Amazon, Google, Microsoft and Nuance — to include Apple, taking specific aim at the tech titan's Siri virtual assistant.
Interesting. A "patent troll?" At first blush, looks like one to me. Bears watching.

ON THE VENTURE CAPITAL COMMUNITY

A staple of Health 2.0 events involves panels of Silicon Valley VCs. Below, an interesting read (particularly in the wake of the persistent troubles at Uber, and the misfortunes of bro'grammer #GoogleManifesto Man):

In December 2010, Sheryl Sandberg gave a talk about women’s leadership in which she mentioned “sitting at the table.” Women, she said, have to pull up a chair and sit at the conference-room table rather than clinging to the edges of the room, “because no one gets to the corner office by sitting on the side.”

Less than a year later, I would take those words to heart. I had been working for six years at the Silicon Valley firm Kleiner Perkins Caufield & Byers as a junior partner and chief of staff for managing partner John Doerr. Kleiner was then one of the three most powerful venture-­capital firms in the world. One day, I was part of a small group flying from San Francisco to New York on the private jet of another managing partner, Ted Schlein. I was the first to arrive at Hayward Airport. The main cabin of the plane was set up with four chairs in pairs facing each other. Usually the most powerful seat faces forward, looking at the TV screen, with the second most powerful next to it. Then came the seats facing backward. I was sure the white men booked on the flight (Ted, senior partner Matt Murphy, a tech CEO, and a tech investor) would be taking those four seats and I would end up on the couch in back. But Sheryl’s words echoed in my mind, and I moved to one of the power seats — the fourth, backward-facing seat, but at the table nonetheless. The rest of the folks filed in one by one. Ted sat across from me, the CEO next to him, and the tech investor next to me on my right. Matt ended up with what would have been my original seat on the couch.

Once we were airborne, the CEO, who’d brought along a few bottles of wine, started bragging about meeting Jenna Jameson, talking about her career as the world’s greatest porn star and how he had taken a photo with her at the Playboy Mansion. He asked if I knew who she was and then proceeded to describe her pay-per-view series (Jenna’s American Sex Star), on which women competed for porn-movie contracts by performing sex acts before a live audience.

“Nope,” I said. “Not a show I’m familiar with.”

Then the CEO switched topics. To sex workers. He asked Ted what kind of “girls” he liked. Ted said that he preferred white girls — Eastern European, to be specific.

Eventually we all moved to the couch for a working session to help the tech CEO; he was trying to recruit a woman to his all-male board. I suggested Marissa Mayer, but the CEO looked at me and dismissively said, “Nah, too controversial.” Then he grinned at Ted and added, “Though I would let her join the board because she’s hot.”

Somehow, I got the distinct vibe that the group couldn’t wait to ditch me. And once we landed at Teterboro, the guys made plans to go to a club, while I headed into Manhattan alone. Taking your seat at the table doesn’t work so well, I thought, when no one wants you there. (When Sandberg’s book Lean In came out, that same Jenna Jameson–obsessed CEO became a vocal spokesperson for it.)

Seven months later, I would sue Kleiner Perkins for sexual harassment and discrimination in a widely publicized case in which I was often cast as the villain — incompetent, greedy, aggressive, and cold. My husband and I were both dragged through the mud, our privacy destroyed. For a long time I didn’t challenge those stories, because I wasn’t ready to talk about my experience in detail. Now I am…
Yikes. Read all of it. (a fairly long read). I ran into it here. Just an excerpt from her forthcoming book Reset.


Interesting, accomplished, scary-smart woman. Same age as my younger daughter.
____________

More to come...

Tuesday, August 29, 2017

NLP, meet NPE: the curious case of Robert Budzinski


Robert Lucius Budzinski, of 1106 Edgewood Drive, Richardson, TX 75081, patent infringement plaintiff asserting to be the "sole inventor" of the Natural Language Processing (NLP) technology alleged wrongfully used by a breadth of major tech corporations. He is reported to be suing a number of them, e.g., Nuance, Google, Apple, Microsoft, and Amazon.

"NLP, meet NPE"

"NPE" is intellectual property legalese for "Non-Practicing Entity."

Otherwise known as a "patent troll." Mr. Budzinski is the owner/"President"/"Director"/"Registered Agent" (and sole "employee") of "Word to Info, Inc.," a private corporation (operating out of his residence) that apparently produces no discernible products or services (a "NPE"). A "company" with no internet presence of any sort -- if you exclude Mr. Budzinski's LinkedIn page:


That's it in its entirety. No employment or education history, no accomplishments or interests, no connections, no nothing. Zip, zilch, nada, nyet.
I did find Plaintiff's Counsel info in one of the lawsuit filings:
Steven R. Daniels
Texas State Bar No. 24025318
FARNEY DANIELS PC
800 South Austin Ave., Suite 200
Georgetown, Texas 78626
Email: sdaniels@farneydaniels.com
Telephone: (512) 582-2828
Fax: (512) 582-2829


James R. Gourley
Texas Bar No. 24050679
CARSTENS & CAHOON, LLP
13760 Noel Road, Suite 900
Dallas, Texas 75240
Email: james@cclaw.com
Telephone: (972) 367-2001
Fax: (972) 367-2002
Attorneys for Plaintiff
Word to Info, Inc.
I routinely search Google news and other internet sources for anything new on "natural language processing" (NLP), looking in particular for stuff relating to the health care space. Recall my prior posts on the topic, here, here, and here.

I ran across this at AppleInsider.com:
Apple's Siri latest target in string of natural language patent lawsuits
By Mikey Campbell


One-man company Word to Info on Friday expanded a string of patent lawsuits over natural language processing technology — active cases involve Amazon, Google, Microsoft and Nuance — to include Apple, taking specific aim at the tech titan's Siri virtual assistant.

Filed in the patent holder-friendly Eastern Texas District Court, Word to Info's suit alleges infringement of seven patents detailing methods of natural language processing. The company asserted the same series of patents against a number of tech industry giants marketing their own voice recognition and virtual assistant solutions.

Specifically, Word to Info is leveraging U.S. Patent Nos. 5,715,468, 6,138,087, 6,609,091, 7,349,840, 7,873,509, 8,326,603 and 8,688,436 in its case against Apple. The IP string covering methods of interpreting natural language input dates back to 1998, when the U.S. Patent and Trademark Office granted the '468 property to inventor and Word to Info director Robert Budzinski...
I did a cursory review of all seven patent filings listed in the article. They total about 2,100 pdf printable pages of dense detail -- most of the content repetitiously redundant.
UPDATE: the most recent (08/18/17) TX court filing -- against Apple -- by this plaintiff here (49 pg pdf). The suit alleges seven "Claims for Action," each of them mapping to the Budzinski patents set forth and linked above.
The "abstracts" from all of the Budzinksi patents, from the oldest to the most recent:
Abstract
A memory system for storing and retrieving experience and knowledge with natural language. The primary components of this memory system include syntactic processes, function word processes, morphology processes, ellipsis processes, concrete and abstract noun word sense number processes, verb word sense number processes, adjective word sense number processes, purpose identification processes, plausibility and expectedness processes, communication processes, context storage processes, and text generation processes. The syntactic processes include word isolation, dictionary look up, and parsing. The function word processes select and evaluate functions associated with function words which are certain: adjectives, nouns, verbs, adverbs, pronouns, prepositions, conjunctions, and interjections. Morphology processes replace morphological words with phrases or clauses composed of function words arid state representation words. Certain nouns, verbs, and adjectives are state representation words. Selecting the word sense number of a state representation word selects the state representation of such a word. Experience and knowledge are stored as clause implying word sense numbers organized into paths in a directed graph.

Abstract
A memory system for storing and retrieving experience and knowledge with natural language through methods and apparatus is disclosed. The primary components of this memory system include syntactic processes, function word processes, ellipsis processes, morphology processes, meaning word sense number processes, purpose identification processes, plausibility and expectedness processes, communication processes, context storage processes, and text generation processes. The function word processes select and evaluate functions associated with function words which are certain words in each part of speech. Ellipsis processes replace unstated words. Morphology processes replace morphological words with phrases or clauses composed of function words and word sense numbers. A word sense number is an address to the meaning of a word. Certain nouns, verbs, and adjectives are meaning words. Word sense numbers are selected to be consistent with the context of the clause containing the words, the context, and stored experience and knowledge. Experience and knowledge are stored as nodes with associated clause implying word sense numbers organized into paths in a directed graph. Nodes in the directed graph have access conditions which determine if a node is accessible on a path. A path has an associated purpose relation which is any concept that labels the path. Purpose identification processes select: purpose relations, experience and knowledge, processes for setting a verb's result states or a state value, object classification paths, and activity selection paths. The communication processes coordinate incoming and outgoing natural language text. Text generation processes generate natural language text from word sense numbers.

Abstract
A memory system for storing and retrieving experience and knowledge with natural language through methods and apparatus is disclosed. The primary components of this memory system include syntactic processes, function word processes, ellipsis processes, morphology processes, meaning word sense number processes, purpose identification processes, plausibility and expectedness processes, communication processes, context storage processes, and text generation processes. The function word processes select and evaluate functions associated with function words which are certain words in each part of speech. Ellipsis processes replace unstated words. Morphology processes replace morphological words with phrases or clauses composed of function words and word sense numbers. A word sense number is an address to the meaning of a word. Certain nouns, verbs, and adjectives are meaning words. Word sense numbers are selected to be consistent with the context of the clause containing the words, the context, and stored experience and knowledge. Experience and knowledge are stored as nodes with associated clause implying word sense numbers organized into paths in a directed graph. Nodes in the directed graph have access conditions which determine if a node is accessible on a path. A path has an associated purpose relation which is any concept that labels the path. Purpose identification processes select: purpose relations, experience and knowledge, processes for setting a verb's result states or a state value, object classification paths, and activity selection paths. The communication processes coordinate incoming and outgoing natural language text. Text generation processes generate natural language text from word sense numbers.

Abstract
A memory system for storing and retrieving experience and knowledge with natural language through methods and apparatus is disclosed. The primary components of this memory system include syntactic processes, function word processes, ellipsis processes, morphology processes, meaning word sense number processes, purpose identification processes, plausibility and expectedness processes, communication processes, context storage processes, and text generation processes. The function word processes select and evaluate functions associated with function words which are certain words in each part of speech. Ellipsis processes replace unstated words. Morphology processes replace morphological words with phrases or clauses composed of function words and word sense numbers. A word sense number is an address to the meaning of a word. Certain nouns, verbs, and adjectives are meaning words. Word sense numbers are selected to be consistent with the context of the clause containing the words, the context, and stored experience and knowledge. Experience and knowledge are stored as nodes with associated clause implying word sense numbers organized into paths in a directed graph. Nodes in the directed graph have access conditions which determine if a node is accessible on a path. A path has an associated purpose relation which is any concept that labels the path. Purpose identification processes select: purpose relations, experience and knowledge, processes for setting a verb's result states or a state value, object classification paths, and activity selection paths. The communication processes coordinate incoming and outgoing natural language text. Text generation processes generate natural language text from word sense numbers.

Abstract
A memory system for storing and retrieving experience and knowledge with natural language through methods and apparatus is disclosed. The primary components of this memory system include syntactic processes, function word processes, ellipsis processes, morphology processes, meaning word sense number processes, purpose identification processes, plausibility and expectedness processes, communication processes, context storage processes, and text generation processes. The function word processes select and evaluate functions associated with function words which are certain words in each part of speech. Ellipsis processes replace unstated words. Morphology processes replace morphological words with phrases or clauses composed of function words and word sense numbers. A word sense number is an address to the meaning of a word. Certain nouns, verbs, and adjectives are meaning words. Word sense numbers are selected to be consistent with the context of the clause containing the words, the context, and stored experience and knowledge. Experience and knowledge are stored as nodes with associated clause implying word sense numbers organized into paths in a directed graph. Nodes in the directed graph have access conditions which determine if a node is accessible on a path. A path has an associated purpose relation which is any concept that labels the path. Purpose identification processes select: purpose relations, experience and knowledge, processes for setting a verb's result states or a state value, object classification paths, and activity selection paths. The communication processes coordinate incoming and outgoing natural language text. Text generation processes generate natural language text from word sense numbers.

Abstract
A memory system for storing and retrieving experience and knowledge with natural language through methods and apparatus is disclosed. The primary components of this memory system include syntactic processes, function word processes, ellipsis processes, morphology processes, meaning word sense number processes, purpose identification processes, plausibility and expectedness processes, communication processes, context storage processes, and text generation processes. The function word processes select and evaluate functions associated with function words which are certain words in each part of speech. Ellipsis processes replace unstated words. Morphology processes replace morphological words with phrases or clauses composed of function words and word sense numbers. A word sense number is an address to the meaning of a word. Certain nouns, verbs, and adjectives are meaning words. Word sense numbers are selected to be consistent with the context of the clause containing the words, the context, and stored experience and knowledge. Experience and knowledge are stored as nodes with associated clause implying word sense numbers organized into paths in a directed graph. Nodes in the directed graph have access conditions which determine if a node is accessible on a path. A path has an associated purpose relation which is any concept that labels the path. Purpose identification processes select: purpose relations, experience and knowledge, processes for setting a verb's result states or a state value, object classification paths, and activity selection paths. The communication processes coordinate incoming and outgoing natural language text. Text generation processes generate natural language text from word sense numbers.

Abstract
A memory system for storing and retrieving experience and knowledge with natural language through methods and apparatus is disclosed. The primary components of this memory system include syntactic processes, function word processes, ellipsis processes, morphology processes, meaning word sense number processes, purpose identification processes, plausibility and expectedness processes, communication processes, context storage processes, and text generation processes. The function word processes select and evaluate functions associated with function words. A word sense number is an address to the meaning of a word. Word sense numbers are selected to be consistent with the context and stored experience and knowledge. Experience and knowledge are stored as nodes with access conditions and with associated clause implying word sense numbers organized into paths in a directed graph. A path has an associated purpose relation which is any concept that labels the path. Text generation processes generate natural language text from word sense numbers.
Yeah, gave me MEGO as well. Beneath each abstract are dozens of equally dense enumerated "claims" ostensibly setting forth the putative (and obtuse) "operational details" of the "invention." The old jibe "if you can't dazzle 'em with brilliance, baffle 'em with bullshit" comes to mind.
BTW: I just asked Siri on my iPhone 6s (multiple times) "Is Robert L. Budzinski a patent troll?" "She" couldn't accurately process the query. All I got was repeated irrelevant gibberish.
Also of note: I never once encountered in these patent documents the phrase "computational linguistics" (or the word linguistics, for that matter).


Moreover, checking the body text and index of this authoritative reference book I have studied and heretofore cited, no mention of a "Budzinski." You would think that a guy purporting to be the "sole inventor" of a huge chunk of applied AI/NLP (worth potential billions) would have an academic and professional rap sheet a mile long, one that would show up in a boatload in scholarly citations.

Maybe I'm missing something. Not for lack of extensive searching.

UPDATE

You keep digging, using various name/word/phrase combinations and permutations, eventually you find something.


1981. Looks like he worked for T.I. Obviously some kind of microelectronics guy (engineer?). Link to the above paper here. I'd eventually run across an IEEE link citing him as as a co-author on three papers (including the above) around that time (the other two are firewalled in pdf at IEEE).

Still have yet to unearth a C.V. Still smells like "NPE," notwithstanding that his cred has to be up just a notch in my eyes, although technical cred is a separate issue from patent trolling.

There's a ton of reportage on patent trolling, (which has been recently asserted to "cost the economy" $80 billion a year), e.g.,
'THE ULTIMATE PATENT TROLL'
In the words of technology reporters.
MORGAN BASKIN & JACK DENTON


Founded in 2000, Intellectual Ventures "has earned a special brand of hatred in the business world as the ultimate patent troll. It doesn't delay your flight like United, buffer your movie stream like Comcast, or shellac your shrimp with oil like BP. Rather, it hoards ideas." It "goes around to companies and says: 'Hey, you want to protect yourself from lawsuits? We own tons of patents. Make a deal with us. Our patents will not only cover everything you're doing in your business, no one will dare sue you." "It then wields this intellectual-property portfolio—the world's largest—like a weapon. Companies can either pay up or face a lawsuit.”…
__

Back to Budzinski...

"PLAINTIFF DEMANDS A JURY TRIAL"

But of course he would.

In 1991 I took a gig as the Technical Editor (pdf)  for "CSI" ( Computational Systems, Inc., of West Knoxville TN). It was a small recent startup founded by two PhD industrial-electrical engineers who were alumni of a larger competitor (TEC). We designed, built, and marketed digital Fast Fourier Transform (FFT) industrial analyzers. We were staffed up with hardhat assembly and C language programmers writing tight code straight to the chip clocks.

A great, fun job. Heady stuff. Kind of an Appalachian "Silicon Valley" loosey-goosey culture producing then-state of the art portable industrial digital tech (we were just across the pike east of Oak Ridge, where I'd spent the prior five and a half years in a radiation lab writing code). My department was an "artsy" Mac shop, basically an in-house ad agency doing magazine-quality 4-color digital pre-press.

Well, our competitor sued us for patent infringement, based on asserted purloined IP (essentially a de facto "non-compete" "leave your brain at the door" case). We were killing them in sales (owing more to our assiduous customer service and support than to our technologies per se). They retaliated.

And they won. Jury trial. Crippling monetary judgment. Only one juror had beyond a high school education. [Technology For Energy Corporation v. Computational Systems, Inc., et al., (E.D. Tenn., Fed. Cir.) (Patent infringement litigation)]

I survived the ensuing layoffs.

Wasn't a patent troll thing, but I will never forget the turmoil. In July of 1992 I left to move to Las Vegas in the wake of my wife's transfer and promotion to manage QA at the Nevada Test Site nuke cleanup project.

I then took an analyst/LAN manager job with the Nevada Medicare QIO (HealthInsight, then known as Nevada Peer Review). I recall during my first stint (of three) with them a discussion one day concerning an ortho doc who did more than a quarter of all hip jobs in the state. Owing to the relatively high efficacy of his px, he sought to patent his surgical "method."

Lordy. The blowback was pretty severe. I don't think he ever went through with it.

BTW, a good, quick read on intellectual property fundamentals and issues:

"We all create intellectual property. We all use intellectual property. Intellectual property is the most pervasive yet least understood way we regulate expression. Despite its importance to so many aspects of the global economy and daily life, intellectual property policy remains a confusing and arcane subject. This engaging book clarifies both the basic terms and the major conflicts surrounding these fascinating areas of law, offering a layman's introduction to copyright, patents, trademarks, and other forms of knowledge falling under the purview of intellectual property rights..."
I finished this book in short order. Highly recommended survey look into the breadth of sub-topics of IP: patents, copyrights, trademarks, and trade secrets, as well as relatively minor tangentially overlapping areas such as fashion design, internet domain names, and celebrity exclusive use of "likeness." Good discussion of the evolved histories and global differences in IP scope and enforcement.
Recent item: General Mills just (rightfully) lost an IP lawsuit wherein they tried to obtain exclusive trademark rights to the color "yellow," on the grounds that market identification of Cheerios was/is inextricably bound up with it in the product's packaging. Consumers might otherwise be "confused."
The jokes just write themselves.
Jokes aside, we might ask, what is the proper scope of intellectual property ownership enforcement in the law?
If there were only one man in the world, he would have a lot of problems, but none of them would be legal ones. Add a second inhabitant, and we have the possibility of conflict. Both of us try to pick the same apple from the same branch. I track the deer I wounded only to find that you have killed it, butchered it, and are in the process of cooking and eating it.

The obvious solution is violence. It is not a very good solution; if we employ it, our little world may shrink back down to one person, or perhaps none. A better solution, one that all known human societies have found, is a system of legal rules explicit or implicit, some reasonably peaceful way of determining, when desires conflict, who gets to do what and what happens if he doesn’t…


Friedman, David D.. Law's Order: What Economics Has to Do with Law and Why It Matters (p. 3). Princeton University Press - A. Kindle Edition. 
I have not the slightest doubt that, were I to query Budzinski's patent attorneys, I'd be met with harrumphingly indignant, derisive (albeit utterly self-serving) paternalistic pushback lecturing down to me about the Altruistic Nobility of their tireless work in defense of "the little guy."


A planet populated by more than 7 billion contending people is a complicated place. Our technologies are complicated, ever-moreso. Necessarily, our laws are increasingly complicated. And, among the most complex among them are the laws going to IP. Nonetheless, to the extent that IP filings are woefully obtuse, dense documents riddled with verbose, redundant, vague, ambiguous, often internally inconsistent jargon-language, they seem to principally only serve the lawyers whom they mostly enrich.**
** A similar observation can be made in general about legislation and the regulations they authorize. Overcomplexity and lack of clarity in law inevitably beget legal challenges, where the courts have to try to clean up the messes.
For one thing, perhaps "Loser Pays" IP litigation reform might serve to mitigate the excesses in this area. (Ohhh... the poignant wailing that that notion produces in tortland!) I also have to wonder how IP litigation firms' expenses are covered prior to judgments or settlements? Is this stuff typically done on "spec?" And, are there the equivalent of "VC/investment funds" bankrolling these actions? I seriously doubt that our NLP boy Rob is paying out of pocket by the billable hour.

 THE NPE AT SILICON VALLEY HBO


LOL.

CODA

Pretty interesting read at The New Yorker:
Who Owns the Internet?
What Big Tech’s monopoly powers mean for our culture.
By Elizabeth Kolbert
...Thirty years ago, almost no one used the Internet for anything. Today, just about everybody uses it for everything. Even as the Web has grown, however, it has narrowed. Google now controls nearly ninety per cent of search advertising, Facebook almost eighty per cent of mobile social traffic, and Amazon about seventy-five per cent of e-book sales...
And Rob Budzinski wants him a piece of all that. "Rentier?"

Relatedly,
Google Just Proved That Monopolies Imperil Democracy, Not Just The Economy
Barry Lynn and his team of anti-monopoly researchers were fired by a think tank after criticizing the search giant.


WASHINGTON ― For the past decade, former business journalist Barry Lynn has used his perch at the New America Foundation to warn politicians and the public that a new era of corporate monopolies threatened not only American workers, but also democracy itself.

Lynn was just proven right: New America has fired him as head of its Open Markets program along with his team of about 10 researchers and journalists, after they called for an antitrust investigation of the think tank’s largest longtime donor, Google…
Naked Capitalism is also on the story: "New America Foundation Head Anne-Marie Slaughter Botches Laundering Google’s Money, Fires Anti-Trust Team at Eric Schmidt’s Behest."

OH, AND, ANOTHER THING,

apropos, read this in my AAAS Science Magazine:
Fostering reproducibility in industry-academia research
Science  25 Aug 2017:
Vol. 357, Issue 6353, pp. 759-761
DOI: 10.1126/science.aan4906


Many companies have proprietary resources and/or data that are indispensable for research, and academics provide the creative fuel for much early-stage research that leads to industrial innovation. It is essential to the health of the research enterprise that collaborations between industrial and university researchers flourish. This system of collaboration is under strain. Financial motivations driving product development have led to concerns that industry-sponsored research comes at the expense of transparency (1). Yet many industry researchers distrust quality control in academia (2) and question whether academics value reproducibility as much as rapid publication. Cultural differences between industry and academia can create or increase difficulties in reproducing research findings. We discuss key aspects of this problem that industry-academia collaborations must address and for which other stakeholders, from funding agencies to journals, can provide leadership and support…

Barriers to Sharing
Efforts to promote reproducible research have varied. One widely supported strategy is to increase the availability of data produced in studies, along with computer code written to clean and analyze data. Publishers and funders have instituted policies mandating data deposition or data management plans; however, success has not been uniform.

There are disincentives to open sharing of information. For academic research, rewards come from public presentations and publications that lead to recognition within the community, grants, and tenure. The emphasis on publications to reap academic rewards means that academic researchers can be reluctant to release information or even to fully describe their work. In industry, publishing is typically not a high priority; the goals are the provision of a product (whether goods or services) that will outstrip competitors and provide monetary rewards to investors. The need to obtain patents or maintain trade secrets to protect intellectual property (IP) can provide a strong financial incentive to not disclose or share information. Corporations see relatively little advantage to releasing data for research purposes, so any nonzero risk of consequences (even if only hypothetical) can be sufficient to shut down such efforts…

The Creative Commons license CC0 (which waives all rights of data authors) is attractive as it does not require data sets to have a “provenance” trail and can thus ease automated mining of data. However, lack of provenance tracking in CC0 creates challenges for data evaluation, interpretation of analyses, and accreditation of data generators, thus making CC-BY (in which author attribution is required) attractive. Both continue to be discussed as aspirational goals.

Irreproducible research wastes time, money, and resources. Academic researchers, universities, and other institutions, industry, funding agencies, and editors all have a role to play in raising research standards and creating an environment of trust between communities.
Hmmm... For one thing, I refer you to this book I cited a while back (scroll down):

____________

More to come...