There might be a few ways to do this, but this describes a method I found that works. The goal here is to return features as a json payload of attributes + geometry and it's up to the client to do something with them.
Here's how to author the tool in a Python Toolbox:
Make an output parameter with datatype = "DEFeatureClass" and parameterType = "Derived". I played around with GPFeatureRecordSetLayer and GPFeatureLayer, but DEFeatureClass was the only way I could get it to work. In the execute function, create a feature class in the in_memory or scratchGDB workspace. It doesn't really matter which one, but in_memory was much faster in my testing. You can make the feature class from scratch, or if you are getting your features from an existing feature class, make a feature layer with arcpy.MakeFeatureLayer_management, and then use arcpy.CopyFeatures_management to copy the feature layer to a feature class. Return the output parameter as follows: parameters.value = fc_path, where fc_path is the entire path to the feature class, like in_memory\<feature class>. Right before you create this feature class though, check for its existence and delete it if it exists, otherwise if you run it consecutively it will still be there from the previous run.
Here's how to publish the service:
Important! Even if you don't reference any existing data, you have to enable the feature "Allow data to be copied to the site when publishing services". This can be found in ArcGIS Server Manager under Site, GIS Server, Data Store, Settings. You can turn this on temporarily just for publishing the service and then turn it off and the service will still work, but you can't publish this gp service without it - you will get an ERROR 001488. Run the tool, right click on the gp result and and Share As GeoProcessing Service. There's really nothing special you have to set here, other than the normal metadata and parameter descriptions, but I recommend configuring the service to run asynchronously because you never know how long it will take to create the file. Be aware of the service's feature limit in the Parameters section. The default is 1000. If your result exceeds this, the json response will include "exceedTransferLimit": true and inexplicably the features list will be empty. You do not get the first <feature limit> features, you get no features at all. So be aware of this. When you execute the task from the REST endpoint, you will get a json payload with data type GPFeatureRecordSetLayer that includes the spatial reference, the fields schema, and the features, each with attributes and geometry. The Web AppBuilder very nicely adds this feature set as an operational layer to your layer list.