Group Avatars

That’s a two part answer

Size: That’s a BuddyPress function. They have instructions for that https://codex.buddypress.org/getting-started/guides/customizing-buddypress-avatars/ The issue you’ll run into is wanting to do it for groups and not for members. You’ll have better luck asking over in the Buddypress forum

Square: You’ll have to add some custom css to your child theme. This might not get all instances of the group avatar, but try this for the group directory.

#buddypress #groups-list .item-avatar {
border-radius: 0 !important;
}

#buddypress #groups-list .item-avatar.rounded img {
border-radius: 0;
}