John Elder Robison, author of Look Me In The Eyes, talked about his asperger’s syndrome going undiagnosed. A great book and in it he talks about the adults being the only ones that interacted with him due to his social skills.
People with autism can be challenging. However, as adults, we can handle it. When behaviors get on our nerves, we need to be the ones that keep them from isolation.
Their behaviors are not a choice to be naughty. They can be organic and they can be learned. They may make comments that are insensitive because they cannot predict how it will be taken.
Those behaviors do not need to be ignored. However, we need to make sure the person behind those behaviors are also not ignored. You do not know how important you might be to someone without positive relationships.