• jarfil@beehaw.org
      link
      fedilink
      arrow-up
      1
      ·
      11 months ago

      That’s my point. They claim to reduce misrepresentation, while at the same time they erase a bunch of correct representations.

      Going back to what I was saying: fine tuning doesn’t increase diversity, it only shifts the biases. Encoding actual diversity would require increasing the model, then making sure it can output every correct representation.

      • Even_Adder@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        3
        ·
        11 months ago

        It doesn’t necessarily have to shift away from diversity biases. I think with care, you can preserve the biases that matter most. That was just their first shot at it, this seems like something you’d get better at over time.

        • jarfil@beehaw.org
          link
          fedilink
          arrow-up
          2
          ·
          11 months ago

          I guess their main shortcoming was the cultural training set. I’m still unconvinced that level of fine tuning is possible without increasing model size, but we’ll see what happens if/when someone curates a much larger set with cultural labeling.

          The labels might also need to be more granular, like “culture:subculture:period”, or something… which is kind of a snakes nest by itself.