This release has major changes related to containerization for the flow execution. It provides a way to isolate flow execution on containerized infrastructure like Kubernetes. For more detail, please refer to this design doc: https://azkaban.readthedocs.io/en/latest/containerization-design.html
9018fa2c (HEAD -> master, tag: 4.0.0, origin/master, origin/HEAD) Flow deterministic ramp-ups for imagetype and containerization
17bb970c Added jobtype based filtering for containerization ramp-up
0ae21638 Adding dependency for javax.el as it needs to be upgraded to 3.0 for hibernateValidator dependency (#2792)
adce7491 Allow multiple copies of the same dependency file in Thin Zips. (#2754)
57286fb5 Merge Flow and server logs for containerized execution. (#2790)
d1a6efe6 ApiGateway support for ramp-up mechanism from POLL to CONTANERIZED dispatch
b285244c Give job props precedence over cluster props (#2786)
2ab81c19 Added mechanism to rampup Containerized from POLL dispatch
566bb1d8 Resolve symbolic links for jar files added to HadoopSecurityManagerClassloader (#2782)
7750f80f Marking flow as PREPARING from DISPATCHING as POD creation request is submitted (#2784)
ab50ddcf Adding an option to provide flow container resource requests from flow parameter (#2780)
270defa2 Changed start status to READY instead of PREPARING for poll based dispatch
8024fada Add probes from the podTemplate to the podSpec
e10a0226 Merge cf-engine-k8s to master (#2772)
f49d8654 create-all-sql-*.sql is missing in some distributions
40625b17 Fix hyperlink to Containerization Design doc. Missed one leading "`" (#2767)
09e72927 Add Image Management APIs to index of documentation for easier access. (#2764)
de5aa520 Updated image management API documentation. (#2766)
4a12ddec Delete unused "schedule options" files (#2755)
c2b1634f Marking flow as PREPARING from DISPATCHING as POD creation request is submitted. (#2759)
a9a8b8f0 Fix handling of null executionId in ExecutorApiGateway (#2752)
e6febcc2 Fetching all the hadoop tokens from single method and moving call to specific method as responsibility of each implementation (#2746)
8d219eee Fixed image link for flowchart in imageMgmtAPI.rst (#2745)
902c6bd0 Containerization Design Doc (#2727)
16a6fd5b Bring back "Flow override properties to also override job properties" (#2632)
f6678214 Add pipeline Level 3: block all jobs until the previous flow has completed (#2515)
7d770b2a Flow container cherry-pick from cf-engine-k8s (#2742)
87b1d561 Build Azkaban against Hadoop 2.10. (#2729)
fb742825 Add executor api calls for containerized flows (#2715)
b3c5bb63 Rename polling DB lock to include schema name (#2720)
ce35c92f Add secret mounting to kubernetes dispatch (#2711)
e3d8e5be Prevent kill action from leaving executions stuck in KILLING status. (#2691)
635a3fcc Restructuring code for HdfsAuth so that there can be multiple implementations of HdfsAuth and we can have an option to choose any implementation. (#2710)
670be395 Add ability to mount secrets in the PodSpec builder (#2702)
cc46cd3d Updated AzKubernetesV1SpecBuilder.java to add hostPath volumes (#2689)
8c7926ce Add a date-time utility method and do a few refactors. (#2688)
e8b4b6a9 Moving common HadoopSecurityManager code to AbstractHadoopSecurityManager so that all the versions of HadoopSecurityManager can reuse this code. (#2685)
51fe9b24 Solidified FlowContainer logic with integration testing. (#2687)
7a18f7bd Add TLS support to ExecutorApiGateway (#2679)
438930b7 Expose Azkaban Executor Server configuration to the cluster router (#2675)
4c9993fd Fixing the name of Property class for Containerized Dispatch. Earlier it was ContainerizedExecutionManagerProperties. (#2680)
1d68caff Added version_set table DDL and JdbcVersionSetLoaderImpl class for interaction
09154223 Add periodic cleanup of containers (#2671)
5ca7d39f It contains integration with service spec builder and service creation. (#2674)
a10ff5d4 Added class for creation of kubernetes service object using template (#2667)
1c0642c6 It contains integration with pod spec builder and pod creation. It also contains delete methods for pod and service. (#2669)
6941dad2 Use CredentialProvider instead of CredentialProviderWithKeyStore unless needed. (#2664)
259b2510 Remove duplication in embedded flow name. (#2661)
f7036b61 Job specific cluster components dependencies should take precedence over those of jobtype plugin. (#2660)
e33546d5 BDP-3878: Magnet on flow ramping regardless failure (#2652)
65b25bfc Introducing version set and added skeleton for version set loader
d41c9880 add a timer metric for flow startup delay (#2650)
7f86e683 Added builder class for creation of Kubernetes pod-spec
1121c83e Introducing FlowContainer. (#2647)
543797fb Merge robin branch into master (#2644)
1e8c84c1 Adding dispatch logic for containerized environment (#2635)
15222785 Expand the length of some fields related to Ramp Feature (#2630)
b3c26fb4 Add a new CredentialProviderWithKeyStore interface which extends CredentialProvider interface with KeyStore specific APIs. (#2629)
8fb2667b Updaing dependency jopt-simple to 5.0.3
d0b519cb Add get/set KeyStore methods in CredentialProvider interface. (#2625)
aee22b0a Removing duplicate code in ExecutorManager and ExecutionController (#2623)
8b65539a Fixed the PDT-->PST time change tests. They should pass until atleast 2029 (#2626)
5375629a Make MetricsManager more robust to changes in metrics reporter Ctor
43177fae Fix issue 917 and other startup issues
94f9ac84 Revert Flow Override feature #2241 (#2620)
b60eb055 Patch6 (#2616)
e9f17894 Add CSRF token validation for modifying project permissions
f5e294d2 Fix startup errors in some environments (#2614)
b2173ea1 Adding utility method to check PUSH and PUSH_CONTAINERIZED dispatch m… (#2607)
c05f2339 Add flow execution source to flow properties (#2591)
3d07ead0 Add metric to measure the executor polling frequency. (#2599)
4386e776 Fixes failing WebUtils unit test (#2612)
fae5b513 Update Travis CI configs. (#2609)
55b8eb9c Fix grammatical errors (#2605)
603afd4b Fix grammatical errors (#2606)
b0aad21a Enable security logging via AzkabanEventReporter (#2608)
98eb6ca1 Restore flow lock message after project upload. (#2600)
9a50b73a Fix flaky queue duration condition in JobRunnerTest (#2604)
734121f6 Fix grammatical errors (#2601)
f8a72d08 Fix grammatical errors (#2602)
783f88d4 Adding enum for Dispatch Method. It can be PUSH, POLL or CONTAINERIZED (#2603)
0fc132fa Formatting only: apply Save Actions on some files (#2596)
ad72f4ac Flow override properties to also override job properties (#2241)
5b8eb08f Revert "Bring back Multiple Hadoop cluster (Robin) changes and fix LocalFileSystem leak. (#2585)" (#2592)
76537a35 Add job props to uniquely identify a job in mapreduce tags (#2593)
457957ca [UI] Improve readability of long job property values (#2580)
953b8729 Logging improvements for job callbacks (#2583)
efef0034 Fix typos (quiting -> quitting) (#2581)
497d4cdb Add metrics to API endpoint requests. (#2590)
2217f85d fix bug in calculating flow kill and pause durations,and job kill and preparation durations (#2578)
c79c8ce8 Remove null from startup-dependencies.json while writing and reading … (#2588)
1a78e1c1 fix typo for couple documents (#2584)
dd075323 Bring back Multiple Hadoop cluster (Robin) changes and fix LocalFileSystem leak. (#2585)
99daa7b5 UI and API changes to prepare Azkaban for OAuth SSO. (#2587)
991238b3 Revert "Add metrics to API endpoint requests" (#2582)
a041bcdc (origin/limit_flow_retry_count) Revert "Pass cluster object around instead of cluster id during job parameter construction in JobTypeManager." (#2579)
ece3f081 Support sla settings in execute & schedule API actions (#2523)
d18bdc83 Formatting only: apply Save Actions on some files (#2562)
f2e3b681 Refactoring only: extract SLA parsing to new SlaRequestUtils (#2522)
31bd7422 Add metric NumAgedQueuedFlows: Count flows that are in queue for a long time (#2549)
23430813 Fixed the thread-pool usage when downloading dependencies
31fa0b76 Pass cluster object around instead of cluster id during job parameter construction in JobTypeManager.
604b3791 Add metrics to API endpoint requests
51eac654 Fix handling of Api failures during executor healthcheck
278a3135 Closed the IO streams in the thinzip code path.
1eec4e47 Expose cluster routing exceptions to job logger for easy debugging.
7e3899e1 Migrated to PDL
ec24817f Fix jobrunner fireEvent timing.
5402abf8 Support routing jobs to a single Hadoop Cluster in Azkaban Exec Server.
c4a16c7c Introduce a per-job classloader during job execution.
6e3635c7 Revert "Introduce a per-job classloader during job execution."
5ca11009 Modify fields for Azkaban Event Reporter.
9dc44864 Add Azkaban project event data.
ee385ed8 Make HadoopSecurityManager_H_2_0 non-singleton.
519395c2 Introduce a per-job classloader during job execution.
48bc24f3 Revert "Introduce a per-job classloader during job execution."
3bd23cfc Introduce a per-job classloader during job execution.
687bb999 Remove Teradata jobtype in Reportal, and add idea plugin to build.gradle
5669e77d Changed assertEquals orders and remove a unit test assertion to prevent build error.
6253a370 added attributes for job and flow life-cycle events
4c296d12 Allow jobtype plugins to process job properties before jobs start to run.
2466fbb5 [UI] Correctly indicate when a schedule has SLA rules defined in the Schedules page.
634e202a Extract API endpoint names and some other (web|exec) server configurations into constants.
84dd461c Introduce a concept of cluster component, through which jobs and jobtypes can declare their dependencies.
5a0f1027 Change to Builder Pattern
1a98b4e4 Project Cache Abstraction
9255ef24 Handle exception during executor shutdown due to flow ramp disabled on clusters
dc12ebac Remove direct instantiation and cast to Hdfs DistributedFileSystem
dfc71f1a Improved error messages. Changed annotation for NotNull to Nonnull
98664650 Create an API to modify polling interval of an executor
9edab54d Refactor of Web server metrics.
183baf2c Add thread names for multi-threaded pools
f9d71716 Upgrade Azkaban to the ~latest version of rest.li.