Yes, there is the rss_enclosure_image_size filter which can be used to set the thumbnail size. For example, this snippet will set the image size to “large” “full”:

add_filter( 'rss_enclosure_image_size', function() {
    return 'full';
} );