Imagine your search terms, key-strokes, private chats and photographs are being monitored every time they are sent. Millions of students across the U.S. don’t have to imagine this deep surveillance of their most private communications: it’s a reality that comes with their school districts’ decision to install AI-powered monitoring software such as Gaggle and GoGuardian on students’ school-issued machines and accounts.

“As we demonstrated with our own Red Flag Machine, however, this software flags and blocks websites for spurious reasons and often disproportionately targets disadvantaged, minority and LGBTQ youth,” the Electronic Software Foundation (EFF) says.

The companies making the software claim it’s all done for the sake of student safety: preventing self-harm, suicide, violence, and drug and alcohol abuse. While a noble goal, given that suicide is the second highest cause of death among American youth 10-14 years old, no comprehensive or independent studies have shown an increase in student safety linked to the usage of this software. Quite to the contrary: a recent comprehensive RAND research study shows that such AI monitoring software may cause more harm than good.

  • PotentiallyApricots@beehaw.org
    link
    fedilink
    arrow-up
    24
    ·
    2 months ago

    This is awful. Surveillance is not a replacement for childcare. How many times must people say it. It is also not a replacement for managing employees or any other thing. I hate this timeline.

      • PotentiallyApricots@beehaw.org
        link
        fedilink
        arrow-up
        24
        ·
        edit-2
        2 months ago

        Kids going to school cannot reasonably be expected to have the knowledge, forethought, or ability to protect themselves from privacy violations. They lack the rights, info and social power to meaningfully do anything about this. That’s why it’s exploitative and harmful. Edit: that’s also to say nothing of the chilling effect this is going to have on kids who DO need to talk about something but now feel they have to hide it, or feel ashamed of it. Shit is bad news all around.

        • TexMexBazooka@lemm.ee
          link
          fedilink
          arrow-up
          1
          ·
          2 months ago

          Sooo schools should just provide devices to kids with no monitoring at all?

          There shouldn’t be an expectation of privacy on school/company provided devices, that isn’t how it works literally anywhere. It’s on the parents to teach their children not to use the device for personal reasons.

          Ideally the school machines should be limited to only allowing coursework and limited messaging between classmates and teachers, it’s a tool not a toy.

          Idk I just can’t get upset about this. Kids and privacy is kind of a tough one to begin with, I personally think kids shouldn’t have unregulated access to communication devices at all until like 14-15, maybe.

          • Vodulas [they/them]@beehaw.org
            link
            fedilink
            arrow-up
            16
            ·
            2 months ago

            Yes. There are tons of enterprise tools to lock devices to certain activities. Surveillance is not necessary and will be used to violate privacy, and I am not talking about just on device communication. Remember when companies were caught using their employees cameras without any indication on the device? The suspected benefits of surveillance is not worth the potential harm.

            • TexMexBazooka@lemm.ee
              link
              fedilink
              arrow-up
              3
              ·
              2 months ago

              My company exclusively deploys machines with physical coverings for the camera and hardware disconnects for the mics.

              • Vodulas [they/them]@beehaw.org
                link
                fedilink
                arrow-up
                8
                ·
                2 months ago

                Good! Not all companies do that and I highly doubt school districts in most places will. They tend to be underfunded and understaffed

          • PotentiallyApricots@beehaw.org
            link
            fedilink
            arrow-up
            12
            ·
            2 months ago

            An issue here for me is that the kids can’t op out. Their guardians aren’t the ones checking up on their digital behavior, it’s an ai system owned by a company on a device they are forced or heavily pressured to use by a school district. That’s just too much of a power imbalance for an informed decision to my mind, even if the user in question were an adult. Kids are even more vulnerable. I do not think it is a binary option between no supervision and complete surveillance. We have to find ways to address potential issues that uphold the humanity of all the humans involved. This seems to me like a bad but also very ineffective way to meet either goal.

            • TexMexBazooka@lemm.ee
              link
              fedilink
              arrow-up
              2
              ·
              edit-2
              2 months ago

              We just fundamentally disagree on what rights someone is afforded on a company provided devices. They can’t opt out because obviously not, you don’t get to just opt out of information security policies.

              It would be a different beast if the school didn’t allow you access coursework on a personal machine without installing their bullshit, thats a huge issue.

              • PotentiallyApricots@beehaw.org
                link
                fedilink
                arrow-up
                7
                ·
                2 months ago

                Yeah, I just fundamentally don’t think companies or workplaces or schools have the right to so much information about someone. But I can understand that we just see it differently.

                • TexMexBazooka@lemm.ee
                  link
                  fedilink
                  arrow-up
                  1
                  ·
                  2 months ago

                  I agree on that point, nobody has the right to any information about me except for exactly what I choose for them to know. Speaking from an IT professional standpoint, if I deploy a device, I absolutely have the right to know anything that happens on that device. You have to from a security perspective.

                  That’s why I don’t use any social media on my work laptop. Ideally that’s why social media is blocked on work machines so it’s a non-issue. Kids should understand that concept early, you do have a right to privacy but you also don’t control that device.

              • Hazelnoot [she/her]@beehaw.org
                link
                fedilink
                English
                arrow-up
                4
                ·
                2 months ago

                It would be a different beast if the school didn’t allow you access coursework on a personal machine without installing their bullshit, thats a huge issue.

                That’s exactly how it works at many places. Students can only use a personal device if it’s enrolled in the school’s MDM, which grants them just as much control.

  • kittenroar@beehaw.org
    link
    fedilink
    English
    arrow-up
    18
    ·
    2 months ago

    I suspect the real reason they are doing this is to condition the students to accept oppressive and unjust surveillance in future workplaces.

    • chicken@lemmy.dbzer0.com
      link
      fedilink
      arrow-up
      8
      ·
      2 months ago

      I doubt the school administrators who would be buying this thing or the people trying to make money off it have really thought that far ahead or care whether or not it does that, but it would definitely be one of its main effects.

    • The Doctor@beehaw.org
      link
      fedilink
      English
      arrow-up
      5
      ·
      2 months ago

      Just as securicams in schools in the 90’s conditioned a lot of people to accept on-street surveillance.