It can *sometimes* matter to the beholder, some people prefer average some prefer larger and some prefer smaller. Preferences aren't bad. But if it matters, it probably matters more to the owner than to anyone else. We all want to be accepted for our penis size I think regardless of what we were dealt between the legs.
I believe that size is more of a mental obstacle for men than a physical one. Bigger is always perceived as better but the main issue here is whether or not a man can reach the G-spot with his erect penis. The G-spot is just inside the upper vagina opening and is packed with jillions of nerves empowered to drive a woman insane. So if your penis is at least 3-4 inches when erect you are able to reach this magical place. Since 75% of women reach orgasms not by vaginal intercourse but clitoral /G-Spot stimulation, we men shouldn't be all that worried about its length. The bad news is that we are because it is perceived that bigger is better.
In nudism (an outlook, attitude, state of being), nothing matters. All acceptance. Kumbaya. In nudity (a conscious sexual dynamic), physical attributes and preferences (and more) matter. The problem is that, in social settings, the two sometimes become intertwined (often on nude beaches). A couple of years ago I was hanging out with a girl and we got along very well. I asked her on a date to a local nude beach. We had a great time. Of course the topic of body acceptance (yes, including penis size) came up, as it was her first time at a nude beach. She stated matter-of-factly that I have a small penis and asked me whether I was embarrassed to be seen among the many other guys with larger penises. I said not at all and that such things are of no concern to a nudist. In any event, I said I would like to take her out again and, the following week, I called her a couple of times. She didnt answer. I never heard from her again. On the one hand, one could argue that she was seeing nakedness, not nudism (nothing wrong with that). On the other hand, it was a date and she had every right to consider her own preferences. And so this is my example of how, while nothing matters in nudism, at the same time, in practice, things can be a little more complicated. But even with nudity, apart from a nudist philosophy, its actually quite simple: everyone is still entitled to their own preferences!