A factory, manufacturing plant or a production plant is an industrial site, usually consisting of buildings and machinery, or more commonly a complex having several buildings, where workers manufacture goods or operate machines processing one product into another. The word factory generally refers to a production site where a specific item is produced, whereas a plant refers to a site where a specific process takes place. For example, a site that produces brushes could be called a brush factory, but you wouldn't call it a brush plant. A factory is where things are made or constructed. A factory began as a word as a wholesale goods warehouse run by an owner or hired manager called a “factor.” Places where stuff was made with specific tools and complex processes go back to stone toolmaking, leather tanneries, potters, stone quarries, grain milling, etc.. had other names specific to the work or “workshop.” If it was an operation substantially powered by a waterwheel, it was often called a “mill.” A factory can be as small as a single small room.
A Plant usually means a place where things are not usually constructed, rather it is an area or building, such as where fresh water is produced by purification, or sewage is disposed, or where grain is milled into flour. They’re used interchangeably and seem to be a factor of age of both the speaker and the facility (factory=older plant=more recent “facility” seems to be replacing both.) A “plant” was usually a set of buildings, purpose-built, and originally for power generation, chemicals manufacturing, refining and smelting metals, cement-making, meat packing, food processing, beverage manufacturing, etc. As long as the other person or readers can understand what you’re referring to by the context and modifying words or photos, it doesn’t matter. Factory can be used quite widely too for very different purposes. A factory is usually associated with making discrete parts on machines, and / or assembling parts into products (discrete manufacturing). A Plant is usually associated with continuous physical / chemical (as opposed to mechanical) transformation to produce products (continuous or batch processing). For example, compressing natural gas, petroleum refineries, food products, cement, etc. A Plant is utmuch larger. Some factories have more than one building to conveniently organize production. Plants are ALWAYS with multiple buildings as well as exposed equipment organized as per the production process. Colloquially the words Factory and Plant are sometimes used interchangeably. For example, Cement Factory (batch process), Car Plant (discrete manufacturing), etc.