Anticipatory Conformity & Algorithimic Bias

I am glad we have spent some time thinking about the everyday consequences of data surveillance, and I think that discussing the “Screening Surveillance” films has been a great way to get at the issues (moving beyond mere conceptual analysis) as we start to think about how this ubiquitous reality is shaping our everyday lives. The conversation we had about “A Model Employee” and more recently about “Blaxites” lead to some meaningful insights on your part. Doing some speculating together (imagining different versions of the story or how small plot changes could lead to very different outcomes) has also been away for us to understand what is at stake.

Earlier, Maura did some important highlighting of a few critical concepts from Shoshana Zuboff’s analysis, including the ways machine learning and tech companies have “outrun public understanding” and the key concept of “anticipatory conformity” – which should lead us all to question how we can be sure our beliefs and behaviors are ours (rather than a result of the perception we come by that is ultimately shaped by the algorithms framing our world via the internet).

We also have had a chance to look at the TED talk by Joy Buolamwini:

Buolamwini zeroes in on role that data plays in algorithmic injustice, helping us understand how our human blind spots (i.e. racism) can very well become “baked in” blind spots when data is mobilized via machine learning. She illustrates the flaws that may exist with data sets, and then expands her analysis to include the problems that result from the practices/policies that are shaped by the use of artificial intelligence. How does algorithmic bias lead to discriminatory practices? When algorithms become the basis for things like predictive performance in school, for predictive policing, and for overall societal access (admissions, loans, insurance, etc) – our future(s) are indeed cast by data patterns. Buolamwini also calls for “inclusive code” and inclusive coding practices. As she states, this starts with considering the people writing the code (who is included here?), and the ethics of the design teams who are employed to design new models. Can we “factor in fairness” when the world itself is already rife with inequity, before we scale our systems for efficiency with machine learning? In bringing up the recent case of researcher Dr. Timnit Gebru at Google, I meant to point us all towards both the currency and urgency of these questions.

…And just a reminder to check out Alan’s post which has given us all much to play with regarding AI exploration:

I AI Therefore I Am? Our explorations for this week's #netnarr dive into algorithmic stew. This post written by my pal "Jeremy" https://t.co/JyITIYFRkC
— Alan Levine (@cogdog) March 25, 2021

And also check out this thread:

I urge #netnarr to read not only the tweets (to the tune of Gil Scott Heron) but the full paper for an example of speculative fiction. It’s 2050 for the Post Post Pabdemic University. https://t.co/IkxootKuUY
— Alan Levine (@cogdog) March 27, 2021

Next Thursday

I am looking forward to our time together next Thursday! #NetNarr is pleased to host another Studio Visit with special guest Chris Gilliard. We will engage Chris in a lively discussion of race, data surveillance, and artificial intelligence. Please check out Chris Gillard @hypervisible.

Join us for a special #netnarr & #unboundeq Studio Visit (4/1, 4:30pm ET). Registration in link below:

Race, data surveillance, and artificial intelligence: a discussion with activist/scholar @hypervisible https://t.co/jxqf0Izsau… via @UnboundEq
— Mia Zamora (@MiaZamoraPhD) March 28, 2021

Please register for this event here.

After we connect with Chris, we will jump back into our regular #NetNarr Zoom room to hear from Tom, who will be pathfinding based on Ruja Benjamin’s “Catching Our Breath: Critical Race STS and the Carceral Imagination” Engaging Science, Technology, and Society 2 (2016), 145-156. It will be an engaging evening!

Finally, just a reminder that we have all agreed to make Thursday April 8th a free evening. I hope you will take the “breather” to refresh and recuperate from this dense and challenging stretch of time.

See you all Thursday!

Dr. Zamora

Announcements

Our Wrap Up! April 28, 2021 11:24 am

So the time has come for us to synthesize our Post-Pandemic University dreams and nightmares in a brilliant collection of tiny tales. So much can be addressed with the power of a story, and so much can be laid bare with speculative fiction. Our collection of PPU tiny tales will be speculative-fiction-as-literature. Your micro- fictions will explore possibilities in a society which may not yet have been enacted, but are latent. Any technology or fantastical element in speculative fiction ought to have roots in what our current species can already do, or is on the road to being able to do. It seems this whole course has been a lens on issues that are present (or near present) in the world of higher education. How is the work of “being educated” changing? And in what ways are our expectations for education changing too? How is “the enterprise” of knowledge-production shifting as we rely so much more on technology to pursue this ambition? How will the Post-Pandemic University shape our understanding of what it means to learn/grow/evolve or be “trained” for society? I sincerely look forward to reading each and every one of your speculative PPU tales. As the collection comes together in the Arganee Journal, we will then be able to consider the implications that emerge from your collective response to these pivotal questions. When we meet for our final class, each of you will have a moment to share your work-in-progress. We will treat our last class like an “Author’s Chair” round-robin. ….What is an “Author’s Chair”? This protocol is typically the final component of writer’s workshop. During this 10 minute block, individual students share their writing with an audience of classmates. After sharing, peers provide structured feedback. This process supports authors in their journey towards becoming flourishing writers. So…this means everyone will take the chair at some point to share their concept-in-progress, or walk us through a rough draft of their tiny PPU tale. I look forward to the vision, the imaginative meanderings, and the inspiration!!! See you all soon :)…

Speculative Microfiction Stories: The Specs April 21, 2021 6:41 pm

We have worked our way through our topical discussions that should inform your final summary task. Here we get creative and will be crafting microfiction stories that can paint a picture about what the experience looks like in a near future Post Pandemic University. Speculative fiction implies the kind of stories we saw in the Screening Surveillance short films (Blaxites and A Model Employee) that are set in a time that could almost be now where surveillance technologies act on characters in a dystopian fashion. You can find more examples in our Net Mirror Library from 2020. For what we mean by microfiction, think back to our first studio visit with Laura Gibbs — and here you will be writing them as text but limited to 500 words or less. For some excellent examples as models, see the collection of Telling Stories: On Culturally Responsive Artificial Intelligence. This will be published in another of our course sites where previous NetNarr students have published their work, the Arganee Journal (look for a demo in class, it is actually another WordPress like site for publishing articles). The specifications for your stories include: The 500 word limit. You will want to compose somewhere else, and editor, Google docs, etc, or just paper. It’s a challenge to write short. If you look at the Telling Stories: On Culturally Responsive Artificial Intelligence stories, you see the main character is often introduced and put into a setting in the first sentences.The Setting. This story takes place in a fictional but plausible Post Pandemic University (PPU). It can be a physical location, digital, or both. One Image to Represent the Story. You will need to find an open licensed image (or use your own) that can work as a metaphor or representation of your story.The character. A story needs a character, heroic or not. Imagine who it might be at a PPU? Student? Teacher? Staff? Employee? What is their role? What is their goal at PPU? What are their strengths? Weaknesses?The Arc. Stories have a shape. Refer to the idea of Pixar’s Story Spine (but you all are writers so you likely know this). Your story launches with the character in their everyday world. In Blaxites, Jai was relaxing at home on her phone. In A Model Employee, Neeta walks to her job at the restaurant. But then something happens, that changes everything, and in this case, it is something triggered or influenced by technology, surveillance, tracking of data. Jai’s prescriptions were cut off because the medical system was surveilling her social media habits. Neeta was coerced into wearing a tracking device to keep her job. Because of this, something else happens, then something else, and at the end of the story, the character’s world is forever changed.The Surveillance Technology. Here is the key part. You need to identify some aspect of the operations of PPU where your character’s life is affected by being surveilled. It is not limited to monitoring devices or hidden cameras, think about all the possible transactions at PPU where your characters actions, information, data might be surveilled or affected by artificial intelligence. Maybe it is using the learning management system, accessing library resources, tracking by learning analytics, campus mental health providers, athletics, predictive technologies for admissions, retention, amount of time until graduation, even alumni giving, campus purchases and bookstore transactions. Because it is an obvious topic and in the news, we want you to think of something other than exam proctoring (but it might include tutoring?). We will have in class activities the next two sessions to “workshop” you through the process. In our next class, we will brainstorm possible ideas for surveillance technologies. We will also make use of a highly relevant and interactive tool Co-designing with Speculative Data Stories especially it’s Story Creation Tool. Anyone who works or studies in a university is likely to encounter a huge range of technologies in the course of their daily activities. These are technologies that help us communicate, collaborate and create, as well as store data, keep track of activities, assess performance, remind us of due dates, check for plagiarism, and more. Many of these technologies offer the capacity for increased surveillance, and some are already being used to monitor and/or quantify some of our activities. The very ubiquitousness of these technologies means that we don’t always have time to think about how they might be affecting and even changing our practices and identities as learners, teachers, managers and/or technological facilitators. The aim of this Speculative Data Stories Creator is to give you a space to step back and think about technology in the present, and what this might mean for Higher Education in the future. We want you to be able to not only think about what is happening, but to allow yourself to imagine possible future scenarios, which might involve existing or speculative technologies. Rather than limiting yourself to describing something that has actually happened, or that is already in place, we want you to be able to use the medium of fiction to explore aspects of an interaction with technology, or concerns it raised in you, by speculating about what could happen.(Emphasis added as this is relevant to our class!) http://datastories.de.ed.ac.uk/datastories/newstory/ This can help you identifying a possible surveillance technology to base your story on and to imagine all the ways and means that it might affect you. Let your imagination roam in thinking about what a Post Pandemic University experience can look like. And it can be both an idealized positive experience or a dystopian dark one. You aim is to create a compelling short story that can situate what this near future might be like. And it can have an ambiguous ending left for the reader to determine (there is no “happily living after”). Are you ready to tell a story? Featured Image: Conversion. Safety razor plant. Library of Congress Public domain image. Estelle Wilson, one of a New England razor factory’s many women workers, checks completed more…

Institutional responsibility & #againstsurveillance April 17, 2021 12:16 pm

Thanks to all of you for another thoughtful evening with excellent conversations, covering some heavy hitting issues for the post-pandemic university: proctoring technology and software, as well as the institutional role in ensuring equitable and just environments for higher education. Despite the open admittance that your collective hearts felt burdened on Thursday evening (and that you have considerable challenges on your plate – both medical and stress-related), you all “showed up” with a thoughtful engagement of this work. Thanks to Hugo for a thorough overview of the problem of virtual/online proctoring, and the ways it puts students (and educational cultures) at risk. Despite the fact that these services claim to uphold “academic integrity”, they sweepingly assure a culture of mistrust and disbelief, the invasion of privacy, inequity, and an assumption that everyone is out to cheat before they even engage. Where is the call for the development of more authentic forms of learning assessment in the midst of this 32 billion dollar industry surge? There is no doubt any such call is muted or hard to hear over the deafening march of market and data-driven solutions. Despite this, there have been notable moments of advocacy that have lead to scaled teach-ins (i.e. #AgainstSurveillance: A fundraiser in defence of Ian Linkletter) as well as on-the-ground policy change that matters for everyone (i.e. University of Michigan Dearborn made a university-wide decision to reject remote proctoring and invest in faculty development instead). This is an example of what can happen as a result of activism, advocacy, and stakeholder engagement, when considering the high stakes of #edtech contracts and decisions at the institutional level. At Kean, we have recently been invited to share our thoughts about the future LMS contracts that the University will make – this might be an indicator of a better direction that is unfolding as we move towards our own post-pandemic university. These issues lead right into Bailey’s thoughtful discussion stemming from Brenna Clarke Gray’s Digital Detox #6: Build Back Better. Bailey organized for some great questions about the concept of “moral injury”. What might students want in their technology platforms? There is the financial factor to consider/worry about as many institutions are going to be looking at shrinking budgets in the near future. It’s hard to understand and easy to generalize how various technology decisions are made. Thankfully, at some institutions there are opportunities for input, though usually not for students. How do students even become aware of what is at stake? What moves can we make to broaden these conversations? I do have in mind some work that can be done at Kean for one. And our class (and the discussions we have had) are a starting point for more. This all makes us start to wonder what might be the long term implications are of not attending to these concerns? Thanks Bailey for casting light on the question of moral leadership, while bringing to light the significance of institutional contexts for higher education’s future. What is next? It is hard to believe that we only two have two more classes to spend together. The close of the semester is fast approaching, and next week we will roll out the final project plan which will be a speculative micro fiction about the post-pandemic university. This tiny tale will be your own unique culmination of the impactful concerns/perspectives you have gained by traveling down varying NetMirror pathways as we have collaborated together in this semester-long #netnarr discussion. But before we start our workshop to launch the micro fiction work (part 2 of next class), we will hear from Gee as our final pathfinder (part 1 of class)! Gee will share thoughts on a “Gen Z and the pandemic” (TED Talk): https://www.ted.com/talks/darcy_dixon_gen_z_how_a_generation_defined_a_pandemic. In addition, she will also have us think about the introvert’s perspective on life during the pandemic: https://lens.monash.edu/@education/2020/09/02/1381125/covid-19-and-education-were-the-pandemic-lockdowns-beneficial-for-some. Please check out this material and blog some thoughts on it for Thursday. Have a good weekend!

Thinking about race and technology in #NetNarr April 10, 2021 10:56 am

As we delve deeper into the realm surveillance in the age of big data and algorithms, we also continue to consider implications for the post-pandemic university. Here are the agenda slides from class last time we met. What a pleasure to kick off our #netnarr class time with a special Studio Visit conversation with scholar/researcher Chris Gilliard. He really opened up our understanding of what is at stake regarding surveillance technologies, and we were able to hear a bit more about digital redlining practices, coded bias, and institutional technology policies. We also heard a bit about who is “on this case” at the close of our visit with him, including the ACLU, Future of Privacy Forum, and Chris did indicate that there is some movement on the federal level. There is no doubt that awareness on the ground, and advocacy from citizens who push to expand our right to privacy, have had a growing impact on the development of new ed tech tools under the momentum and sway of innovation capitalism. The educational technology sector is now very big business, and educators and learners must know about this boom (and weigh in at this critical juncture) if there is any hope for a more humane understanding of what it means to be “educated”. In part, this is what this #NetNarr class is all about. Everyday citizens are now calling for an increase of the control individuals have over their personal information, and we must continue to assert that our civil liberties should be enhanced, rather than compromised, by technological innovation. Design ethics matter. Thanks to Tom for opening up conversation about the Ruha Benjamin article “Catching Our Breath: Critical Race STS and the Carceral Imagination” Engaging Science, Technology, and Society 2 (2016), 145-156. “How might we craft a justice-oriented approach to technoscience?” asks Ruha Benjamin. How do data driven technologies and existing sociotechnical systems encode, naturalize, or amplify hierarchies of power and difference? By alluding to the “carceral imagination”, Benjamin asks a critical question: can we understand the role of technology in the upholding and remaking of contemporary racialization and inequality? How has the carceral ideology of surveillance tech infected higher ed? or health care? or the entire education sector? These are big questions, and she begins to answer them with important case studies in her work. What is clear in her research is that this is not one big conspiracy, but rather, she reveals in detail how technology is enlisted in, and productive of, contemporary dynamics of confinement, classification, and control. What we can see from this work is that the carceral sphere is not limited to the domain of the prison proper, but rather extends to encompass a much broader “carceral continuum” of technologies, economies, policies, and social formations premised on racialized and class-based exclusion and punishment. Benjamin urges us to “denaturalize” the sedimented histories of our present-day sociotechnical systems and cultivate—and design for—a different and more just future. What can we do? In thinking further about the future of education (the post-pandemic university), this week we had another impactful #netnarr and #unboundeq Studio Visit with neuroscientist and professor of pathophysiology and biomedical ethics – Mays Imad. The conversation was one that centered on trauma-informed pedagogy and hope. If we are to center our humanity in any effort moving forward, then this conversation is a critical piece of that understanding of what it means to learn. I welcome you all to view this meaningful conversation which definitely helps understand the stakes and impact of our teaching/learning practices in the context of here and now: https://youtu.be/RBEl5G4nA94 What is next? #againstsurveillance We are now on the “homestretch” of our “pathfinding tour”. Next Thursday we will have two pathfinding conversations, lead by Hugo and Bailey. Hugo will consider the article by Jason Kelley, Students Are Pushing Back Against Proctoring Surveillance Apps, Electronic Frontier Foundation, September 25, 2020. Bailey will open up a conversation based on Brenna Clark Gray’s Digital Detox #6: Build Back Better,TRU Digital Detox, January, 2021. Please read both articles and blog for Thursday your thoughts and reflection on the material. Thank goodness for the small break this past week, no? For me, it was just what I needed to catch a breath! It is Spring-in-full-gear now, and I hope you have been enjoying the blossoming world outside. See you all soon for more thoughtful conversation. Sincerely, Dr. Zamora

Next Thursday

Leave a Reply Cancel reply