The .ecw format is compressed, meaning that when it is accessed, the file will have to be uncompressed, then read. I am not sure if it is uncompressed when the service starts and as long as it is running, the uncompressed data is held in memory, or if each request to the service results in uncompress, read, export image then send to client. In either case, you will get better performance with a map service that has a cache generated. The trade-off will be the storage, processing and time resources required to generate the cached tiles, depending on the area and number of scales needed. The projection cannot be changed, so any data you overlay on the map cache will either need to have the same projection or it will be reprojected, which could impact performance. Another option, but most of the items above apply- use the imagery in a mosaic dataset, then create and build overviews. You will need an image server license to serve a map service that has a mosaic dataset.
If you can convert the .ecw into uncompressed and multiple images, then add those to a mosaic dataset (with overviews built), that would probably have the best performance. The smaller the size of the image that is being read, the faster it will load. This is more apparent when zoomed in to larger scales. From there, you can decide if you want to use a cached map service to decrease draw time.
